Show HN: Immersive Gaussian Splat experience of Sutro Tower, San Francisco

It's really cool, but I can't shake off an "uncanny valley" feeling, with all of the small quirks in the geometry. What I think I'd be interested in is a post-processing step where this splat is automatically converted to a 3d model that approximates each component, only falling back to the point cloud if there's no simple shape that fits the observation at a particular location.

c-fe• 1 year ago

This is close to the idea of convex splatting (recent paper) in which convex shapes are used to approximate these real 3d objects as they are better suited than gaussians

lbeckman314• 1 year ago

https://convexsplatting.github.io/

TIL — very cool work!

thot_experiment2• 1 year ago

If you haven't seen Tunnel Vision (same author) please do yourself a favor and watch it. Dude does some fantastic projects.

pinoy4203• 1 year ago

Great video if you like watching videos from the front of BART which is pretty boring tbf. Much better videos of locos travelling through the alps

akanet14• 1 year ago

Dude

thot_experiment2• 1 year ago

Context defines content.

whatever1• 1 year ago

The fidelity is amazing !

pinoy4203• 1 year ago

Can we see the images it was made with

CamperBob2• 1 year ago

What blows me away is that R/C drones can operate that close to those antennas. Frequency differences don't tend to matter much once you go past 100,000 watts.

xrd• 1 year ago

So gorgeous.

FlamingMoe• 1 year ago

Beautiful, the visuals combined with the music gave me quite a nostalgic feeling for the city where I once worked daily but haven't visited in years.

null0pointer• 1 year ago

I had the same feeling. Excellent work!

yapyap• 1 year ago

I know this from the Watchdog game

JKCalhoun• 1 year ago

The wife got me a Sutro Tower shirt one year for Christmas. This was when OTA digital television rolled out. When the rest of San Jose seemed to be taking down their TV aerials or letting them fall apart I was nerding out: purchasing a new one and mast sections from Rat Shack to set up one on our house.

Scanning the spectrum to pull in KQED, etc. The first Austin City Limits I saw in HD blew my mind.

I think all but one TV station came to the South Bay by way of Sutro. Quite a reach.

atarian• 1 year ago

very cool, reminds me of watchdogs. where's the music from? it's very calming

akanet14• 1 year ago

It's actually a mix my girlfriend made, but the main track is https://soundcloud.com/martinlandh/moonlit-serenity

diogocp• 1 year ago

What about the track in your video "I finally made it to the top of Sutro Tower"? It's beautiful!

The video: https://www.youtube.com/watch?v=A7-3CKulsCc

Samin100• 1 year ago

Wow! I'd love to read a more in-depth blog post describing how to create one of these myself, and maybe even contribute my own splats to a collaborative library for iconic landmarks. I could see interactive splats being added to Wikipedia for popular locations.

akanet14• 1 year ago

I give a bit more color in the twitter thread https://x.com/fulligin/status/1892685973731061937

wonger_• 1 year ago

https://xcancel.com/fulligin/status/1892685973731061937

corysama• 1 year ago

https://reddit.com/r/GaussianSplatting/ has been slowly talking about the subject for a while now. There are probably several articles and vids in the search bar.

If you want GS news, https://radiancefields.com/ reports a lot of advances all the time.

BoredPositron• 1 year ago

Pretty much the same workflow as photogrammetry take a lot of images/videos and put them in one of the SOTA gaussian splatting tools.

sm_park• 1 year ago

There is an app you can try https://scaniverse.com/. it splats using your phone's gpu.

LeoPanthera3• 1 year ago

Broadcast television is amazing, and I'm so sad it's dying out. OK, maybe we don't need to tune in at 6 to watch the latest episode of "Friends" anymore, but for any kind of live events - news, sport, politics, having high-definition video you can pull right out of the air without having to worry about paying for data, latency, or bandwidth limitations, is amazing.

For certain applications the internet can never compete with "broadcast".

perching_aix4• 1 year ago

? Is over-the-air TV broadcast not encrypted and compressed to oblivion in the US? Cause it definitely is here, and you're expected to pay for a decoder card, except for a small handful of channels.

vlovich1232• 1 year ago

It’s not encrypted but compression and reliability of the signal is a mixed bag. It’s not encrypted because congress mandated it as a condition for privatizing the analog bandwidth (ie there’s a carve out for public digital and broadcasters have to broadcast publicly accessible signal just like they had to on analog)

LeoPanthera3• 1 year ago

It's not encrypted, no. And the picture quality is very good. Not 4K, but good quality 720 or 1080, depending on the channel.

ghostly_s• 1 year ago

Are OTA channels subscription based where you are (where)? Or the "decoder card" is just some middleware crap you buy once? Our OTA TV has always been ad-supported, they just moved the same channels to digital though in larger markets there are now quite a few new low-overhead licensed syndicated content options, presumably due to the cheaper cost of air slots.

spps11• 1 year ago

very cool! always wondered what that tower was haha, now i know!

IshKebab2• 1 year ago

This does not work at all on Android Chrome. The about dialog flies in super slowly, continually re-layouting the text, and there's no "small cube" and no way to dismiss the about dialog either.

The background looks tantalising but maybe a little more testing is needed... (e.g. for the most common browser/OS in the world).

akanet14• 1 year ago

Wow surprising, do you have any console errors? I tested extensively in Android chrome. To dismiss the modal you can tap in the margin.

IshKebab2• 1 year ago

I'm not sure how to see console errors on mobile (without the remote dev tools anyway).

I tried again today and it's still really slow but I do get the controls this time so I was able to use it. Very neat!

This is on a Pixel 8, Chrome 113.

8n4vidtmkvmk• 1 year ago

Worked ok for me on Android Chrome. Pixel 7

mhandley• 1 year ago

Worked for me on a 2.5 year old OnePlus Nord 2T, both in Chrome and Firefox. Not a high frame rate, but perfectly usable, even on this pretty old mid-range phone.

moondev• 1 year ago

It runs excellently on my nothing phone 2. What phone are you on?

rd• 1 year ago

Does anything like this exist for just flying around cities (not SF, anywhere) in general? Would love to experience what a drone sees even if it's in a limited area.

dag11• 1 year ago

More conventional mesh-based photogrammetry options include:

- https://maps.google.com (satellite view)

- https://earth.google.com (also in browser, possibly better camera controls for what you want)

- Bing Maps (3D flyover mode, more stale data in my experience)

- Apple Maps satellite view (only on macOS/iOS)

- Google Earth VR[1] (requires a Windows PC and a VR headset that can connect to it)

- Microsoft Flight Sim 2020/2024 (requires a beefy Windows PC, uses Bing Maps plus a lot of other enhancements and rendering goodness. Most lifelike "feels like I'm there" but not true to earth)

I'm not aware of splat-based city photogrammetry aside from one-offs like this but I'd love to learn if there's any such projects!

[1] https://store.steampowered.com/app/348250/Google_Earth_VR/

whycome• 1 year ago

There are a couple okay apps on the Meta Quest that use the Google Earth API and you can fly around. It's neat. Wooorld. Fly. EarthQuest.

wst_• 1 year ago

I think it would help to be able to move up and down (ex. space and shift keys) without changing the camera angle. This seems a better solution, to me, than existing pan with 3rd mouse button.

akanet14• 1 year ago

You can use Q and E

michaeloder• 1 year ago

Fantastic work. This is one of the best gaussian splats I've experienced. Especially in regards to the distant objects and sky. I was surprised at how many more details I could perceive in the VR mode. I couldn't spot the "easter egg" until I switched over.

legitster3• 1 year ago

Completely unrelated, but aerial views of San Francisco blow my mind with how under-zoned the city is.

One of the most desirable places on earth to live and it's on a small peninsula. Yet it's a sea of single-family homes as far as the eye can see.

The distance between Sutro Tower and the "Downtown" SF is less than the distance between the Brooklyn Bridge and Central Park. But could you imagine if that space was filled with 2-3 story townhomes?

cbeach• 1 year ago

One of the most desirable places on earth to live and it's on a small peninsula. Yet it's a sea of single-family homes as far as the eye can see.

Ever wondered if it’s desirable BECAUSE it’s not a dense urban jungle?

kaonwarb• 1 year ago

Yes... but the climate and geography alone would make it highly desirable at, say, 10x density.

gcapu• 1 year ago

Maybe you're right, but we'll never know. It would be great if they allowed some sections to develop so we can test it out. To me it is a desirable location because of the companies, not the lifestyle.

mhh__• 1 year ago

There's a difference between urban jungle and (say) nice 5 story European city buildings.

Londons problem for example is that they tried to be clever and cheap at the same time in the 60s and now we're stuck with it.

xvedejas2• 1 year ago

Well, I desired and moved to SF exactly because it's the closest thing to a dense urban jungle that I could find in California. I even dream of moving to a denser part of the city one day, once I can reasonably afford it, but those parts are so in demand to be pricier.

tshaddox• 1 year ago

There are lots of places that are not dense urban jungles and are not desirable, so I'm certain your explanation is insufficient.

skyyler2• 1 year ago

No. It is the good weather capital of the country.

Mild winters, mild summers.

Not too much rain.

No serious threat from tornadoes or hurricanes.

That’s a very big draw and it wouldn’t go away by making more dense housing, even if the rest of the peninsula was developed like Manhattan.

legitster3• 1 year ago

SF is kind of neither though. If you spend any time in the city, it definitely feels overstuffed. The houses may be cute and small but they are stuffed with roommates. The city is stuck in nostalgia and ignoring that it's bursting at the seams. It's like wearing clothes that are too small and pretending you are still skinny.

mindwork• 1 year ago

Could it be not desirable because of single-family homes, but rather because all lucrative and high paying jobs are located here? And it's proximity to the Valley? Also there is much more events and gatherings happen than in the Valley.

screye• 1 year ago

No? Paris, NYC, Tokyo, Singapore, Shanghai, London are all dense urban jungles. Doesn't stop them from being desirable.

rz2k• 1 year ago

It is easy to argue that Manhattan is far more livable than San Francisco due to the layout and highly convenient zoning, even before taking the obvious transportation advantages into account. San Francisco's advantage is the climate and beautiful natural setting.

Considering the advances in seismic technology made over the past fifty years, it is a shame that much faster upgrades to the real estate have not been encouraged.

Hammershaft• 1 year ago

Bullshit. It's desirable because it's the tech capital of the planet. Is New York undesirable because it's developed density?

renewiltord• 1 year ago

Haha, no, it’s desirable because of geography and things to do. This is proven by the fact that large numbers of people are moving into expensive Mission Bay housing which has no single family homes by it.

adamiscool88• 1 year ago

There would be 3-4X the people, yet still the same amount of roads, services, and public utilities. Why is densifying always seen as some unalloyed good? I constantly see this pitched as a plan for housing problems without the basic consideration of whether human beings thrive in such an unnatural environment.

If you're going to force-densify anything, why not actual low-per-capita population areas [0] and develop mass transit, so North America can have the successful China city-tier model [1] with spread-out opportunities instead of cramming everyone together in one place.

[0] https://en.wikipedia.org/wiki/File:California_population_map...

[1] https://en.wikipedia.org/wiki/Chinese_city_tier_system

m-ee• 1 year ago

Public schools are closing due to lack of enrollment. Transit agencies are cutting back from low ridership and lack of fare revenue. If housing costs were low enough for more people to move in affordably it could be a boon for the city.

legitster3• 1 year ago

I would not say that dense city living is not without its downsides. But if people need to work in these cities, they may as well get to live closer. And if you are already living in an apartment, it's not that much different to live in a 6 story apartment building vs a 4 story one.

There would be 3-4X the people, yet still the same amount of roads, services, and public utilities.

That's the point! Per capita, it should be cheaper to live in cities because infrastructure goes so much further. And if you are arguing for better mass transit, you will have to build many, many more miles if you also want to encourage people to sprawl.

Although I think the strongest case for allowing cities to get dense is it allows greenbelts and less dense areas closer to the city. You can build a big dense city UP and make it easier for people to get out and enjoy nature and farms and etc. Or you can build a city OUT and then it's just desolate city for hours around.

FredPret2• 1 year ago

I enjoy both low and high density living.

The argument in favour of density is that if you increase density, then you also decrease the average distance that people have to travel until they get somewhere interesting, like a job or a shop.

Vehicle-delivered utilities like garbage collection, package deliveries, and mass transit get more efficient, and the same goes for tunnel-delivered utilities like fiber internet and water.

San Francisco is economically one of the world's most impactful cities; it'd be good for all of us if there was more of it. You get all sorts of interesting multiplier effects when you put lots of a certain kind of person in one place.

- all the theater kids in one town: LA

- all the bankers: NYC, London

- all the computer people: SF

TylerE3• 1 year ago

Have you actually checked on those Chinese cities to see how they’re doing? Many are literal vacant ghost towns because it turns out people don’t want to live in the middle of nowhere.

pchristensen• 1 year ago

It is currently being force "low-densified" by restrictions. If those restrictions and force were removed, it would densify itself due to market demands. It would be much, much less forced than the current paradigm.

protocolture• 1 year ago

Mass transit isnt a silver bullet either. Here in Brisbane, they standardized around a narrow gauge designed to pull cart loads of timber down mountains. They duplicate it where they can, but effectively its a city of technical debt. Theres a maximum size of train we are already at, and a maximum number of trains per minute we can sustain.

So we have a decent mass transit system but its not far from peak capacity, and most of what the government has been doing is hacking around that. Trammish busses, cross river rail etc.

So we need to attack the issue from the other side too. We have a weirdly non dense central region, largely due to single issue anti development voters, who dont want apartment buildings right where they should be (on top of mass transit hubs). Instead the inner suburbs are littered with 1950s character homes, battleaxed once for massive profit.

We can take significant load off of a system close to a decade from collapse by simply removing outdated zoning.

And the way the council here operates, utilities and road upgrades necessitated by development are borne by the property developer. So there's really near zero cost in approaching things this way. And they have also used priority approvals, where if a certain amount of floor space of a development is earmarked for light commercial, they can cut a few years off approval time. So theres absolutely no reason not to, as the big residential buildings grow, they grow their own services and utilities.

Ericson2314• 1 year ago

Lol my first thought too, as a native San Franciscan who now lives in New York.

It's a real travesty.

bbcc90• 1 year ago

true. See here for a map: https://sfplanninggis.s3.amazonaws.com/hub/BIGmap.pdf

I live in the that space (between tower and the city) and the local neighbourhood group (HANC) is ridiculously NIMBY. rezoning is happening but it's slow going...

parentheses• 1 year ago

My criticism is aimed at the glacial and mostly mishandled infrastructure projects. That is one of the big reasons for zoning changes taking so long.

I hope the US gets its act together and learns from exemplar infrastructure projects around the world.

physhster• 1 year ago

And horribly outdated and poorly built single-family homes for the most part...

fuddle• 1 year ago

That's super cool, I like how they explain what each antenna is used for.

mcstempel• 1 year ago

wow, this is wonderfully made

aqueueaqueue• 1 year ago

Beautiful. Slow on my phone, but it is doing alot!

toephu2• 1 year ago

Ten television stations, three FM radio stations, and 20 wireless and mobile communications users (i.e. law enforcement agencies, taxi cabs, school buses, wireless internet, etc.) rely on Sutro Tower antennas to transmit signals over the air to the entire Bay Area.

roughly• 1 year ago

For some reason it never occurred to me that Sutro was still a live radio tower - it’s such an SF landmark that I think I just assumed it was decommissioned or something.

bastawhiz• 1 year ago

This is great! Back in 2009 or so, I took dozens of high-resolution photos with my digital camera from the observation spot at Sutro Tower (towards the city, not the tower), and combined the images together in Microsoft Photosynth [0] to create an astoundingly high resolution point cloud of the city. I started with lots of zoomed-out photos, then took an overlapping grid of photos at various zoom levels. I wish Photosynth was still around; I'd love to look at the result again.

[0] https://en.wikipedia.org/wiki/Photosynth

akanet14• 1 year ago

the tower released some gigapixel imagery from the top at https://explore.sutrotower.com/

DonHopkins• 1 year ago

Decades ago a radio technician friend of mine took me up the elevator in the west leg to the top platform, to enjoy the fantastic view of the city. As the caption at the base of the west leg elevator entrance says, it was quite small and cramped indeed, and it definitely was disconcerting when it changed orientation passing through the waist of the tower!

bentt• 1 year ago

That's wonderful, nice work! Love the ambient audio track.

armchairhacker• 1 year ago

How feasible would it be to generate high-quality guassian splats of everywhere, starting with big cities, like Google Maps street view and 3D view?

Then you can make video games and interactive experiences in real-world locations. I doubt the collision handling is there, but you can at least start with something like Microsoft Flight Simulator with low-flying drones.

It would also be great for training AI to generate realistic-looking scenarios. Besides playing and working (ex: VR) in real places, you can in very-realistic liminal spaces. (And it's training on public areas, so less ethical issues.)

akanet14• 1 year ago

the collision handling part is actually not that hard to back into from the splat data. i think HQ splats of everywhere is something we'll start seeing in the next decade! we haven't quite solved scaling issues, LoD, and streaming for splats but the early work is promising

jg0r3• 1 year ago

Could you direct me to resources about collisions and splat data?

I have an extreme interest in deriving measurements from splat data of vegetation, as it tends to reconstruct thin planes like leaves far better than other traditional SfM techniques.

ipsum2• 1 year ago

They already are using NeRFs for Street View. (NeRF is similar to a Gaussian splat with different rendering trade-offs) https://blog.google/products/maps/sustainable-immersive-maps...

keepamovin3• 1 year ago

That is so cool! I really wanna do a project like this just for fun to understand it

rsch• 1 year ago

That looks rad.

One bit of feedback: don't move the camera if someone clicks one of the circles, that is super disorienting. There is also a bug that if a drag to move the camera happens to end on a circle, the popup opens.

ghfhghg• 1 year ago

It doesn't seem to load on Android Chrome or Firefox. Maybe a hug of death?

mortenjorck• 1 year ago

As a child of the 90s, I see this as one of those rare, genuine examples of the “museum in cyberspace” imagined by the futurists of the day. Thank you for keeping the dream alive!

biofox• 1 year ago

Glad I'm not the only one who thought of that. On opening it, my mind went immediately to the 3D virtual tours in Encarta.

aa-jv• 1 year ago

Indeed, you might be interested in the Artificial Museum project, which has managed to realize that ol' cyberpsace dream all over the place:

https://artificialmuseum.com/

For example, check the Vienna map .. so many interesting locations!

gertrunde• 1 year ago

In the same vein, it reminds me of the 1980's Domesday project, which had some sections that were similar to this, although given that it was published in 1986 , it was pretty much point and click to move between static photographs.

[1] https://en.wikipedia.org/wiki/BBC_Domesday_Project

czbond• 1 year ago

Thank you for posting this - really cool project. Also, thank you for sharing your splat experience and methodology.

gameofby• 1 year ago

Excellent!

chriscjcj2• 1 year ago

I had the privilege of spending about 30 minutes at level 6 in May of 2023. It was as spectacular as you might imagine. I would say that if you have even the slightest fear of heights the whole experience would be a nightmare.

arjvik• 1 year ago

How’d you get this opportunity?

chriscjcj2• 1 year ago

I work in broadcast TV in San Francisco and am very good friends with one of the engineers who is responsible for the care and maintenance of some of the facilities up there. We talked about him taking me up there for ten years before we finally got around to it. :-)

keepamovin3• 1 year ago

This is great but please implement mouse look! It makes it way more immersive and is the lingua franca of FPS interactions.

akanet14• 1 year ago

You can free look with the right mouse button

keepamovin3• 1 year ago

Right, or with multitouch gestures on trackpad but I want true mouse look. Better. Pleeease!

You can gate it with pointer capture API but please do it!!!!

archagon2• 1 year ago

My favorite structure in SF. I hope I get the chance to visit the top deck someday.

mihirsahu• 1 year ago

This is beautiful. I didn't know about gaussian splatting, seems like really cool tech

smithclay• 1 year ago

Fun fact: NBC does not broadcast from Sutro tower, but San Bruno…

Which means effectively zero over-the-air reception in parts of the Mission. Lesson learned during a Super Bowl party the 2010s.

jmux• 1 year ago

This is really cool, nice work!

Something I’ve been looking for for a while is an interactive view of SF from above - I think it’d be cool to experience the verticality of SF and see how all the different hills relate to each other (and gmaps/earth just isn't cutting it).

this is actually pretty good for around the panhandle, but if anyone is aware of something like that for the whole city please let me know!

nickvec• 1 year ago

This is awesome. Have been enamored with Sutro Tower since I moved here a few years back. Love that you can see what the different antennas are for as well.

scottrogers86• 1 year ago

love it. love this city, too <3

01HNNWZ0MV43FF• 1 year ago

30 MB! Very cool! It runs about 0.5 FPS on my iGPU but alas... good work anyway

jasonjmcghee• 1 year ago

Very interesting. It runs seemingly flawlessly on my 3.5 year old iphone.

jcarrano2• 1 year ago

Runs perfect on my iGPU, no idea how many FPS but surely enough.

deanmoriarty• 1 year ago

Very beautiful and nostalgic, so well done.

I used to live nearby and my favorite “urban” hike was going up Glen Canyon, up and down the two Twin Peaks, loop through Sutro Forest and then go back to downtown walking down 17th. Loved those weekend walks.

teeeg• 1 year ago

Ha yes love that loop, so strange to see someone else tying it all together! Not quite as good but when I lived in the sunset — grand view to golden heights along the ridgeline then down to Laguna Honda trails and up to twin peaks and out sutro forest to Parnassus.

boguscoder• 1 year ago

Looking under the surface has its own charm

dmazin• 1 year ago

San Francisco is just so beautiful. There's a serenity to it that I can't quite put my finger on. The way the fog waterfalls over the hills when you take the train in from the south... the view from Sutro. The gum tree trove near the tower.

This captured that serenity.

archagon2• 1 year ago

Vincent Woo (the author) also has some lovely time lapse videos of the fog! (And other bits of the city.) https://www.youtube.com/playlist?list=PLXhk-g2kj99k6FMp2tZ2X...

Lorin• 1 year ago

Sadly the controls are not mobile friendly and I ended up spinning wildly.

kiririn• 1 year ago

Works fine on iOS safari, two finger drag to move

kevthecoder• 1 year ago

The Metaverse Standards Forum has had some activity around gaussian splats recently, for example debating whether it's too early to standardise.

There's a town hall on 5th March with speakers from Niantic and Cesium: https://metaverse-standards.org/event/gaussian-splats-town-h....

The previous splats town hall, and other related talks, are on the videos page (there was another gaussian splat talk a couple of days ago from Adobe). https://metaverse-standards.org/presentations-videos/

Luc• 1 year ago

I wasn't expecting to be able to see through the lattice structure. Amazing amount of detail.

Naively, it seems to me that the many needle-like artifacts further away from the tower could be filtered out?

desdenova• 1 year ago

The help text mentions a "little cube" that would enable AR mode, but I can see no cube.

Touch controls are very weird and don't rotate the view as expected.

andybak• 1 year ago

it so nearly runs on a Quest 3. The next gen of mobile XR chipsets (or maybe even the current gen with less thermal throttling) are going to be able to handle these kinds of scenes with a bit more optimization.

akanet14• 1 year ago

there is still some perf headroom in software to get, too!

zokier• 1 year ago

What is the limiting factor for level of detail here? Is it the source data (too low res camera/too far away), the processing, or getting it rendered in real-time in browser? Is this the full detail, or is this somehow downsampled/compressed version for the web?

akanet14• 1 year ago

The processing and alignment of source imagery, imo

FinnKuhn• 1 year ago

I can also recommend "Tunnel Vision: An Unauthorized BART Ride", which was made by the same author and is a really great documentary film.

It's free on YouTube: https://www.youtube.com/watch?v=3-Jrp6it9Ss

franky47• 1 year ago

"Good morning, and welcome to the Black Mesa transit system" is the first thing that popped in my head when the train started moving.

chadd• 1 year ago

I'm biased bc I worked with the team while there, but I believe Snap's acquisition of PlayCanvas was one of their most underrated. Incredible technology.

cloudfudge• 1 year ago

This brings a meta quest 3s to its knees. It's almost so bad you can't quit it, and the video lags 15 seconds, which is very disorienting to be immersed in (it can make you fall down). Shame, since it looks gorgeous.

poutrathor• 1 year ago

what part is the culprit : the meta quest or the 3D implementation from the website ? On a classis laptop, it behaves well

PaulHoule2• 1 year ago

The quest is underpowered (it's basically a midrange Android phone you wear.) More efficient coding or a simpler model would help. Inside the Scaniverse does something similar with a high framerate but the models are simple and don't look very good.

I had to power cycle mine to get out, but boy was the view great despite the motion sickness.

ladon86• 1 year ago

The Quest's Snapdragon GPU, like most mobile GPUs, uses a tiled rendering [1] architecture.

The basic technique for rendering gaussian splats is kryptonite for this architecture, essentially implementing every worse practice for rendering on a mobile GPU:

* Tons of overdraw (overlapping splats)

* Tons of alpha blending

* Millions of splats in the distance generate a lot of tiny triangles resolving to a single pixel

* Long thin splats in the foreground generate triangles that cover multiple tiles

These are all the ingredients you need to bring a mobile GPU to its knees! Any desktop GPU (including most laptops) will be far less sensitive to these issues, even if it's not very powerful. It's a fundamental issue of architecture rather than one of raw FLOPs.

[1] https://en.wikipedia.org/wiki/Tiled_rendering

itishappy• 1 year ago

Weird. Runs totally fine on Intel integrated graphics. 15-20 fps, but starts instantly and is responsive.

PaulHoule2• 1 year ago

Amazing.

I tried it on my MQ3 last night and it was the first thing like that which was photorealistic, but it badly overloaded the MQ3, so it was the closest experience to Sword Art Online I've had yet in VR. (The sky was transparent and my room showed through!) I should have been sitting when I started it but since the frame rate was low and the horizon improperly oriented I could have fallen transitioning to the couch if I hadn't steeled myself to rely 100% on proprioception.

Contrasted to the way too lo-fi Inside the Scaniverse and the bland but cringe Horizon Worlds it's a hit. I gotta try it again in tethered mode.

accrual• 1 year ago

This is very cool. I feel like the technique used (gaussian splatting) gives it a more realistic appearance from certain viewpoints. It almost felt like my monitor turned into a window in the sky looking out over the city for a moment. The illusion falls apart once once drifts too far from the nominal viewing area, but until then, it looks even better than something like Google Earth which renders actual polygons for everything.

jcarrano2• 1 year ago

Not only does it look amazing, it renders super fast even on an iGPU. Feels like magic. A couple of questions: Why do objects flicker when moving the camera? And why do surfaces get translucent when close up?

kevinwang• 1 year ago

very nice, vincent!

pppoe• 1 year ago

Can anyone give some numbers for a more intuitive understanding of the advantages from GS? How large would the file/content be if it is in mesh? Can we get similar rendering FPS?

tuckerpo• 1 year ago

Exponential back-off while zooming in is nice, but maybe reset if scrolling back out.

skeeter_sky• 1 year ago

The background music seamlessly blended with the chorus of Nina Nesbitt's song 'On The Run,' which was playing when I opened the site. I genuinely didn't realize there was any background music at all.

amgee2• 1 year ago

This is fantastic.

I've been trying to build something myself for biological and heritage captures. Could you elaborate at all on the JS decoder method used or resources that ended up enabling you to make it?

I've been playing with Playcanvas React for the past week, would your method integrate with their platform?

ovenchips2• 1 year ago

This application was built on PlayCanvas Web Components: https://github.com/playcanvas/web-components/

It's very similar to PlayCanvas React, but with a fully Web Components-based architecture. You can use the same scripts interchangeably between both frameworks.

amgee2• 1 year ago

I'm assuming you started working on this before their React release, or was Web Components chosen for a specific reason?

I'm trying to make something that can take in real-time data and also handle some XR/AR if possible as well, but I am pretty ignorant beyond html/css/js.

akanet14• 1 year ago

We're trying to get this upstreamed so I actually expect PlayCanvas to talk about it in the future. Here's the reference (cpu) decoder: https://github.com/vincentwoo/blog/blob/master/3d/sutro_towe..., as well as a gpu one: https://github.com/vincentwoo/blog/blob/master/3d/sutro_towe....

For more information on the format itself you can look at https://github.com/nerfstudio-project/gsplat/blob/main/gspla....

Show HN: Immersive Gaussian Splat experience of Sutro Tower, San Francisco

Related Stories