HN user

flor1s

1,138 karma

researcher in (industrial) robotics

Posts7
Comments190
View on HN

Novel view synthesis, so based on some images of a scene, rendering views from positions and angles that were not originally recorded. 3D Gaussian Splatting, besides beating the state-of-the-art in terms of visual quality at its time of release, also has some nice physical properties like having the splats associated with actual points in 3D (obtained through feature extraction).

I guess generative AI using copilot or ChatGPT might overtake model-driven engineering as a time-saving technique, which seems worrisome for the "software factory" industry. At the same time, I doubt it will replace MDE as a formal method (i.e. if you need to prove that your software does what it is supposed to do).

It's weird how mixed quality their books are, and how many overlapping books they have. From Packt I like Sebastian Raschka's books on Python Machine Learning. Other than that from time to time they do have some good "recipes", but hardly any of their books seems worth reading cover to cover.

Weights are just numbers, essentially by using a neural network you are telling the system to "find the best way to represent the scene with a budget of X numbers/parameters". Modern NeRFs like instant-ngp also use some grid representations. I guess Gaussian Splatting is slightly more geometrically appealing because you get points around the surfaces that you are trying to model. These points are however not guaranteed to be exactly on the surface, which additional surface losses try to solve (e.g. NeuSG).

AIST | Research Assistant or Technical Assistant | Onsite in Tokyo, Japan | Python, C++

We are a Japanese semi-governmental research and development institute, our center does research on industrial cyber-physical systems and our team is focused on robotics and automation.

We are looking for candidates with good programming skills who are interested in robotics and have some familiarity with Neural Radiance Fields or are eager to learn.

Research Assistant position (for students who currently live in Japan, at most 14 days/month): https://unit.aist.go.jp/hrd/keiyaku_koubo/2023-icps_0023.htm...

Technical Assistant position (for anyone, at least 3 days/week, but full-time is also an option): https://unit.aist.go.jp/hrd/keiyaku_koubo/2023-icps_0024.htm...

The position is for 4 months starting from December, but might be extended depending on the applicant's performance and project factors.

I'm not sure if it's truly a timeless paper, but "Attention is all you need" by Vaswani et al. has been super influential in recent years. Also "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" by Sohl-Dickstein et al. (about Diffusion Models) and "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" by Mildenhall et al. were hugely influential to me.

Jupyterlab Desktop 3 years ago

tmux + tmuxinator seems like a solution?

Though my workflow right now is mostly SSH + tmux + jupyterlab

In my experience NeRF tends to work better for objects that photogrammetry struggles with, like transparent objects. By using marching cubes you can also export scenes as a mesh. instant-ngp (https://github.com/NVlabs/instant-ngp) has done an amazing job of making NeRF accessible, but you still need camera positions from other software such as COLMAP.

Neural Radiance Fields are a technique from the neural rendering research field, while photogrammetry is a research field on its own. However these are just turf wars and in practice there is a lot of overlap between both fields.

For example, most NeRF implementations recommend the use of COLMAP (traditionally a photogrammetry tool) to obtain camera positions/rotations that are used alongside their images. So this multi-view stereo step is shared between both NeRF (except a few research works that also optimize for camera positions/rotations through a neural network) and photogrammetry.

After the multi-view stereo step in NeRF you train a neural renderer, while in photogrammetry you would run a multi-view geometry step/package that uses more traditional optimization algorithms.

The expected output of both techniques is slightly different. NeRF produces renderings and can optionally export a mesh (using the marching cubes algorithm). Photogrammetry produces meshes and in the process might render the scene for editting purposes.

Oh well... O'Reilly themselves have been losing publishers, a lot of their books (especially containing math) are formatted like crap, their tagging is terrible. Hopefully ACM can make some direct deal with major publishers like MIT Press and Manning.

Pokemon GAN 4 years ago

That's awesome. If I may ask, the data you operate on are still image-like grids or do you operate on more basic data types (e.g. strings)?

Personally I'm also working on an industrial application, using a CycleGAN-based system to augment real world data (e.g. training a network to "paint" an object so we can apply traditional computer vision techniques such as a HSV filter to locate the object). It's quite promising for this kind of application, albeit hard to fine-tune.

Pokemon GAN 4 years ago

Traditional GANs typically have a different problem called mode collapse, in which the generator learns one (or a few) outputs that the discriminator likes. Conditional GANs are given a condition as input and have more chance of overfitting, as the loss function is a combination of the GAN output and a supervised loss based on the condition.

Pokemon GAN 4 years ago

Browsing through their code, it looks like they are using a VQGAN for image generation and ESRGAN for super resolution.

Pokemon GAN 4 years ago

Could you give examples of using GAN in prod? Recently there was a discussion on Twitter about the reduced number of GAN papers in the last few years, and some people mentioned that they did not find adoption in industry.

Not affiliated with this project, but there is a gazillion different variations of GANs. Most just change the adversarial loss to improve the learning rate / quality, but others focus on architectural changes, such as StarGAN, Pix2pix (conditional GAN), CycleGAN, MUNIT, etc. It's really a fascinating field.

There are definitely some areas were biking is amazing, such as Shimanami kaido, Biwa lake and Kasumigaura.

Cycling in Tokyo is a bit of a mixed bag in my experience, some streets have bike lanes, some have a bicycle strip on the main road (though this is often shared with temporarily parked cars), sometimes bicycles need to share space with pedestrians. Signage is often unclear.

I guess calling it bicycle unfriendly is a bit harsh, but at the same time it often feels that bicyclists rank third behind cars and pedestrians.

I think Tokyo still has too many cars (and plenty of two or three lane roads) and Japan as a whole isn't really bicycle friendly. The city is very dense however and the cost of parking spaces makes cars only accessible to upper class families.

At least in The Netherlands we mostly use centralised (gas-powered) heating, so nuclear energy is a tangential factor. Also there seems to be room for improvement in insulation and centralised thermal storage (warmtenet in Dutch). Electric cars seem both like both a problem and an opportunity, because you could charge them while being at the office and partly use their batteries for powering your home at night (in case you don't need a full charge to get to work in the morning).

Regarding rare earths, we will need them regardless of using nuclear or renewables for electric cars, so that seems partly tangential. For wind turbines their reliance on rare earths seems troublesome. It's geopolitically ironic that we (Western Europeans) get our gas from Russia and our rare earths from China.