I just want to serve 5 terabytes [video]

https://www.youtube.com/watch?v=3t6L-FlfeaI

by nobody9999 • 1 year ago

78 34 1 year ago

youtube.com

jonatron• 1 year ago

Text to speech has improved a lot since then.

fifticon• 1 year ago

but the process to serve 5 terabytes hasn't.

flymasterv• 1 year ago

It’s so bad, still.

alexey-salmin2• 1 year ago

What do you mean? 5 terabytes is so little that I forgot how to count that low

vitus3• 1 year ago

Eh, it has, though.

Half of it is about setting up a bigtable for hosting the data. I don't know of any team that's set up a new bigtable in the past 5 years; if you're really going all-in on the cloud ecosystem (per AppEngine) you'll stick it in Google Cloud Storage (where you'll note the file size limit is... 5 TiB) and call it a day. (Also: PCR zones are basically dead.)

The other half is about setting up monitoring, where the mentioned choices are Diplomat (which I don't think anyone's used in about a decade) and Borgmon (which is barely staffed and strongly, strongly discouraged for any new use cases). Borgmon readability hasn't been enforced in years. And again, if you're spinning this up in GCP, just set up some cloud monitoring.

new_user_final• 1 year ago

The video is from 2010

JKCalhoun2• 1 year ago

We love the OG presentation.

redm• 1 year ago

Somehow this has only gotten funnier since it was originally posted..

ur-whale• 1 year ago

Somehow this has only gotten funnier since it was originally posted..

Especially the "only if you think your users are scum" part.

Back in the days, even the asshat in green seem to have tried to uphold some kind of guiding principles.

medwards666• 1 year ago

But ... but ... users _are_ scum, aren't they?

staplung• 1 year ago

That’s no asshat; that’s Broccoli Man! Not the hero we needed but the hero we deserved.

JKCalhoun2• 1 year ago

Yeah, er, or sadder.

leoh• 1 year ago

This tracks with my experiences both at Google and [Google] X.

probably_wrong• 1 year ago

For those who were too young in 2012, here is some background: https://news.ycombinator.com/item?id=29082014

cjs_ac• 1 year ago

Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy algorithms have big constants. Until you know that n is frequently going to be big, don't get fancy. (Even if n does get big, use Rule 2 first.) For example, binary trees are always faster than splay trees for workaday problems.

-- Rob Pike, Notes on Programming in C, 1989[0]

Generally speaking, I feel that the bureaucracy involved in a programming project should be proportional to the scale of the project itself. If the 'getting started' tutorial for your programming language demands that I choose a package name for my Hello World program, you fucked up.

[0] https://doc.cat-v.org/bell_labs/pikestyle

wepple• 1 year ago

I mostly agree, because most proof-of-concepts don’t make it very far, and LLM assisted dev is really strong for that kind of low-barriers experimentation.

The problem is that if you build quick n dirty, but then end up reaching success; you’re in for a bad time or a rewrite. This is where the above example of a single algorithm departs from a whole stack.

Further, one bad “hello world” on a google/apple/amazon domain can make the front page of the NYTimes

bee_rider• 1 year ago

I dunno, if the package name is just some local thing that could be nice. Is “all software is part of a package” ever a nice simplifying assumption?

stickfigure• 1 year ago

Nobody writes hello world for a living. This is a useless metric.

netsharc2• 1 year ago

Recently I went on a trip with a group. Lots of videos and drone footage. Gigabytes of data. The group leader got a 100GB Google Drive space. But if I upload my stuff into it, my stuff doesn't take his 100 GB, but counts against my quota, unless I modify the files to say that $GROUP_LEADER now owns those files.

Well, whatever. I try to download his videos from the web UI. Select all files in directory, wait about 20 seconds, download zip, wait another 20 seconds, finally the file download dialog shows up. It gives me a 2.3GB zip file. I open it, it's just a few files and not the complete directory. It doesn't give me the contents of the whole directory, just files until it reaches 2.3GB, and then silently fails.

Great jaaab, Google!

H8crilA• 1 year ago

They didn't have Borgmon readability.

vachina• 1 year ago

Gdrive’s download as zip never worked reliably for me, and would like you, fail silently with a happy download completed.

For large binaries better to use Google Drive for Desktop (their official client), which is kinda like rsync mount but bulletproof.

ivanmontillam• 1 year ago

Yeah, except for office files (documents, spreadsheets, slides). They become cloud shortcuts, they don't exist on your disk "physically." Don't rely on that for backups.

Google Takeout converts them to actual, solid files.

netsharc2• 1 year ago

I don't see "files shared with me" on the Google Drive for Desktop, do I need to RTFM or watch a YouTube tutorial, because of course it's completely obvious, just like the 100 GB quota issue I described above?

adithyassekhar• 1 year ago

Did you check your browser for google drive site settings?

Google drive has a zip size limit of around 2GB while downloading folders. Usually when you try to download a folder larger than 2GB, it will split these into multiple zip files which all start downloading parallely.

Most browsers block these behind this permission prompt near the address bar "Allow site to download multiple files". If you blink you'll miss it.

nehal3m• 1 year ago

If only Google was the one to make the browser that has to service the request, then they could take this extremely niche use case into account.

zelon88• 1 year ago

My favorite quirk, which they recently did away with, was in Google docs and sheets. They made a custom right click menu that had it's own Copy and Paste buttons. If you tried to use these "Copy / Paste" options, a full page popup would appear telling you to "Use Ctrl + C / V instead".

So someone at Google was unsatisfied with the system right click options, so they remade them. Then someone else at Google probably thought that was dumb and put the popup to remind people that Ctrl + C / V is better? I don't know, all I want to do is move data. But now I have to confront this issue, which takes me away from my actual problem and makes me process some new information that is completely irrelevant. And besides, I already pressed the button. Why didn't you respect that button press? Why did you give me the option to press that button if you didn't want me to use it?

Thanks Google.

alexey-salmin2• 1 year ago

Should've just served it directly from MongoDB. It's web scale.

kburman• 1 year ago

I wish this were a joke, but I’ve actually worked on a production application designed like this.

They use a single table with columns named PK1, PK2, PK3, PK4, and so on, dumping all kinds of data into it. users, orders, addresses, and even blob data. For example, PK1 might be the user ID if the row is a user entry, or PK2 could be the address ID if it’s an address object. And if PK1 happens to contain a #, that might mean it’s a user profile object because apparently that’s how they chose to distinguish it.

As more entities were added, the table just kept morphing into this unstructured, unreadable mess.

There’s absolutely no good reason for this kind of design. It’s just a lack of understanding of how NoSQL systems are meant to be used. The team was told to “utilize NoSQL solutions,” and this is the chaos they produced.

I wish I could unsee it, but the damage is already done. It still gives me nightmares.

cruffle_duffle• 1 year ago

The real fun as you get more experience isn’t the immediate horror show itself but witnessing the process and internal politics that made such a thing happen in the first place.

How did somebody so clearly incompetent get so much political capital that such a system could not only be created but perpetuated and even praised as a good idea.

And then you start to wonder… do the people who praise it secretly know it’s hot garbage or do they genuinely believe their praise? And the answer to that invites even more questions.

And then you get to watch everything else because such a monstrosity doesn’t get built in a vacuum. It’s probably the tip of the iceberg in terms of organizational dysfunction. Who are they hiring (and what does that say about you?). What process enables such things? Why do people tolerate it? Do they know any better? Is it fixable (probably not!) and if so, is it worth it (also no!)

…and of course how the fuck do I get out before I start drinking the koolaid too?

seeknotfind• 1 year ago

Google makes the impossible, hard, and it makes the easy, hard.

I just want to serve 5 terabytes [video]

Related Stories