nobbis

ROS 1 had schema evolution (rosbag migration) 15 years ago

Understood. Thanks for the clarification.

Wait, you renamed Forge (https://forge.dev) released last week by World Labs, a startup that raised $230M.

Is this "I worked with some friends and I hope you find useful" or is it "So proud of the World Labs team that made this happen, and we are making this open source for everyone" (CEO, World Labs)?

https://x.com/drfeifei/status/1929617676810572234

The big difference of course, based on reading this, is that NeRF models the whole light transmission function ("radiance field") whereas this seems to model only the boundary conditions (what the light "hits").

The title of the paper is "3D Gaussian Splatting for Real-Time Radiance Field Rendering". Each rendered pixel weights the contribution of unbounded view-dependent Gaussians. So, no, that's not a difference.

IANAL but I assume so. "Employee of a competing company" (whatever that means) isn't a protected class.

And Matterport will never see them coming, as their employees are banned from using Luma:

Competitors: No employee, independent contractor, agent, or affiliate of any competing 3-D capture company is permitted to view, access, or use any portion of the Service without express written permission from Luma AI. By viewing, using, or accessing the Service, you represent and warrant that you are not a competitor of Luma AI or any of its affiliates, or acting on behalf of a competitor of Luma AI in using or accessing the Service.

https://lumalabs.ai/legal/tos

(Disclaimer: another banned competitor)

Rising rates don't make the dollar stronger. There's no empirical relationship between interest rates and exchange rates.

Yes, although not complete noise – probably empty. Haven't checked but assume there's regularization of the NeRF parameters.

DreamFusion doesn't directly build a 3D model from those generated images. It starts with a completely random 3D voxel model, renders it from 6 different angles, then asks Stable Diffusion how plausible an image of "X, side view" it is.

It then sprinkles some noise on the rendering, makes Stable Diffusion improve it a little, then adjusts the voxels to produce that image (using differentiable rendering.)

Rinse and repeat for hours.

Key step in generating 3D – ask Stable Diffusion to score views from different angles:

  for d in ['front', 'side', 'back', 'side', 'overhead', 'bottom']:
    text = f"{ref_text}, {d} view"

Took a week, as predicted: https://twitter.com/DvashElad/status/1575614411834011651

We rely on Wasm for Metascan (https://metascan.ai/explore) which allows us to use the same rendering code on iOS, the desktop, and the web.

Accessing multithreading is limited as SharedArrayBuffer requires cross-origin isolation to mitigate Spectre. Apart from that, it works great.

Explains the huge remodel of her property here in South Lake.

iPhone/iPad's LiDAR Scanner.

6D.ai wasn’t released until 2018. You may be thinking of Abound, which you wanted to license in March 2017. I don’t know of anyone else that had real-time meshing in 2017.

Interesting - I’ve never observed it kicking in.

Would love to know in which circumstances it’s used. I assume you work for Apple to know this so understand if you can’t share more.

Correlation != causation.

Try out our app, Metascan, to see an example of using TSDF with a multi-resolution GPU hashtable that only stores voxel data near surfaces. Or just skim the original voxel hashing paper from 2013 to understand the technique.

Storing voxel data in an array is a lot simpler. So if it’s not the focus of the research, then why would academics engineer something more complex?

ORB-SLAM2 is SLAM only (similar to ARKit w/o VIO) so only reconstructs feature points - no surfaces. This research is about surface reconstruction.

ARKit doesn’t use LiDAR for camera pose tracking.

TSDF memory isn’t an issue since Niessner et al. (2013).

Their research doesn't just integrate depth maps into a TSDF - it uses NN's to incorporate surface priors.

I don't recall you having similar real-time meshing functionality in 2016-2017, Andrew. Can you show what you had?

As far as I'm aware, Abound was the first to demo real-time monocular mobile meshing: on Android in early 2017 (e.g. https://www.youtube.com/watch?v=K9CpT-sy7HE), and iOS in early 2018 (e.g. https://twitter.com/nobbis/status/972298968574013440).

I said "wanted it" not "wanted to build/finance it." Doubt he's given up wanting it.

A human can always do better: take Google's results, then remove SEO spam/duplicates, extract more relevant snippets, combine results from multiple nearby queries, etc.

Demand exists, but someone has to build it. And it's unclear how big the market is.

Building costs more than money.

I don't agree, plus he's a billionaire.

No, the guy who re-wrote Larry's research code into Python and put it in production.

10 years ago, the original engineer of Google's search engine told me what he now wanted was asynchronous, human-powered search with curated results, e.g. a Google-like interface, but queries cost $5 and take 15 minutes.

Money's no object for him, so he wanted to outsource the filtering, ranking, and interpreting of results. Would be even more useful today (albeit a tiny TAM.)

As far as best practices, Apple has some documentation on how to use their photogrammetry pipeline:

https://developer.apple.com/documentation/realitykit/capturi...

He asked for the “blackbox that takes set of images as an input and produces a 3D model as an output.”

That black box is Object Capture API, which takes a folder of images and outputs an USDZ (or OBJ) file. Model I/O isn’t necessary.

Kudos to Apple for releasing a photogrammetry pipeline that allows web and app developers to build tools like this, with zero knowledge of 3D reconstruction required.

It uses the Object Capture API introduced in the latest version of macOS.

The API allows you to create 3D models from images with just a few lines of code.

https://developer.apple.com/augmented-reality/object-capture

We use MacStadium's M1 mini servers for Metascan's photogrammetry batch processing. They've only been running a few months, but no downtime yet and I'm impressed with MacStadium's customer support, responsiveness, and price.

Not that large. The JS glue code for the VR web page referenced above is 48 KB, including WebGL, WebXR, Fetch, etc.