Xg2xg: Lookup table of similar tech and services for ex-Googlers

by ra7 • 7 years ago

369 126 7 years ago

jarym• 7 years ago

I had never thought about this before but from this repo I can see all these smart engineers at the big tech firms live within their own walled garden (or prison?).

If it weren’t for such a vibrant open source world then they’d really be stuck if they tried to apply their knowledge outside of their companies.

Thanks for sharing the repo!

akhilcacharya3• 7 years ago

They don’t, and if you’ve got the G-factor, pun intended, to get an offer at Google you can get an offer anywhere. Which, honestly, makes this repository all the more confusing to me.

I’m sure there are some people that have lived only within this ecosystem, but that’s the possibly the height of privilege.

badcoderman• 7 years ago

Eh you'd be surprised, I couldn't even get an interview with some places, got denied from others, but now work at Google doing full stack. Every technology I use though is Google internal, and I don't have much knowledge of full stack development outside of Google. Some companies just test your algo / ds / system design knowledge, and sure I do fine there, but any specific knowledge I'm useless. TBH I think I'm not very good at any one language either because I often have to switch

lazyasciiart4• 7 years ago

Doesn't every college hire at FANG fit that description?

q3k4• 7 years ago

I've had companies reject me explicitly because they had bad experience with other googlers. I'm honestly considering dropping it from my CV.

WestCoastJustin• 7 years ago

It is almost the other way around. Many of these open-source projects, have roots with engineers leaving these big companies and wanting to use the same tools, so they create them. Or, by these big companies releasing papers and then folks creating open-source versions. Here's some examples of the latter: Dapper (w/ OpenTracing) [1], Spanner (w/ CockroachDB) [2], Borg (w/ Mesos) [3], Dremel (w/ Drill) [4]. I'm sure there are tons more examples but these are just off the top of my head.

[1] https://ai.google/research/pubs/pub36356.pdf

[2] https://ai.google/research/pubs/pub39966.pdf

[3] https://ai.google/research/pubs/pub43438.pdf

[4] https://ai.google/research/pubs/pub36632.pdf

kyrra• 7 years ago

Prometheus is borgmon. Which is a bit funny as it's used by k8s now. It's also funny as the author left Google, wrote it in 2 years for SoundCloud, then went back to Google.

https://prometheus.io

asdfasgasdgasdg• 7 years ago

A lot more than just ex-FAANG folks would be stuck without the OSS technologies listed in the right column. I daresay, considering much of this software was authored by Xooglers or inspired by Google papers, that it's actually everyone else who would be stuck. The folks who have seen how these problems can be solved at scale would just reimplement the solutions, as indeed they have done.

int_19h• 7 years ago

With Microsoft, this has changed substantially in the past decade, and there are many teams running entirely on public online services (GitHub, Azure DevOps, Travis etc) and with end-to-end open source tooling and dependencies. So it's much less of a technological bubble than it used to be. Probably less so than Google at this point. Can't say about Amazon.

adamnemecek• 7 years ago

We literally talked about this this week

https://news.ycombinator.com/item?id=19576092

Are people really still jerking about a company that's been around for like a 1/4 of a century?

t0astbread• 7 years ago

Sure why not?

And the linked article, while somewhat related has an entirely different message/purpose as this one.

dietr1ch• 7 years ago

How it's the age relevant? I'd guess that the number of engineers and the size of the infrastructure used are far more relevant to just guess how interesting things are.

wbl• 7 years ago

I still have used an editor from a company far older than Google, and would love to learn what life was like on a Symbolics system or the dirty tricks of obsolete networks.

quickthrower22• 7 years ago

1/5 century

sqs3• 7 years ago

Sourcegraph CEO here. Cool to see we are listed as a CodeSearch alternative. We assembled some links to Google internal docs and studies about how they use their Google internal CodeSearch tool. See the links at the top of https://docs.sourcegraph.com/user/search if interested.

tyingq• 7 years ago

Great list, and interesting for non-Googlers too. I'd replace, though:

Google -> Real World

free food -> :(

With:

free food -> free time

QuercusMax3• 7 years ago

Not sure why you think free food is traded for free time. Free food helps me use my time at work more effectively. I usually have breakfast and lunch with teammates, and it's a mix of socialization and work-related conversations. Half an hour in the free cafe > 45-90 minutes going someplace to buy food.

bloomer• 7 years ago

So do you get to go home 45-90 minutes earlier then? I think that is what the original comment was questioning.

andreareina• 7 years ago

Personal preference obviously but I like getting away from work for lunch. After eating if it's nice out I'll actually just sit myself on a bench and chill. 15, 30 minutes doing that is worth so much more to me than leaving work that much earlier.

randomsearch2• 7 years ago

Isn’t the purpose of the free food to make you spend more time at work? Seems logical that the alternative is more time at home.

paxys3• 7 years ago

I'm willing to bet that the average Google employee works as much or even less than the rest of the industry.

googlerx• 7 years ago

Googler here, can confirm. I work around 40 hours a week, sometimes more (as in 42 or 43, not 60) but more often less. Work-life balance is part of the culture.

dekhn• 7 years ago

the free food option at google and now my current startup is a huge time saver. It doesn't come at the expense of free time.

fhrow4484• 7 years ago

Where AWS/GitHub is mentioned, there should be the equivalent Azure stuff, especially for bug tracking, CI/CD, repo subfolder "owners", code review, there's a ton of stuff equivalent or better to buganizer, OWNERS file, etc .

In services there should be vitess since it's both a YouTube and an open source project.

It seem to have a new transactional messaging feature with message sending and Ack: https://vitess.io/docs/advanced/messaging/, which seem to fill a slightly different niche than rabbitmq/pubsub/etc.

myroon5• 7 years ago

I'd love to see this for other large tech companies like Amazon and Microsoft

paxys3• 7 years ago

An equivalent of this for Microsoft is pretty much impossible because there isn't one single way of doing any of these things at the company. Different departments/orgs/teams use different tools, some self-built and some off-the-shelf (including most of the stuff on this list).

nickbarnwell• 7 years ago

I can't speak to Amazon, but at Microsoft the equivalents to the Google offering are going to have the same OSS competitors. Having worked on services there, the only thing I really miss is Kusto; it's now available as an Azure service, but I'm at a company all in on AWS :(

Cthulhu_4• 7 years ago

I like to think that most AWS services were initially developed internally to fulfill an Amazon internal need. But that might be optimistic.

djtriptych• 7 years ago

Came to find critique. Left disappointed :(

sqs3• 7 years ago

We (Sourcegraph) are interested in building an alternative to Critique at some point in the future, or at least enhancing existing code review tools to offer many of the favorite features of Critique.

What are your favorite/must-have Critique features?

(Also would love to jump on a video chat if you or anyone else is interested, and we could live-stream/share it on YouTube for others interested. Email me at sqs@sourcegraph.com if interested.)

sa46• 7 years ago

I left Google about a month ago and use Github now. Things I miss:

- the ability to batch all comments in a single send. GitHub has the problem where replies are immediate and don’t necessarily represent the PR state.

- built-in fixes and lints that show up as comments

- the ability to diff between snapshots instead of losing all previous context every force push to a Github PR.

- Github’s diff is really bad for code moving or changing indentation.

- auto submit if tests pass and you have an lgtm. It’s frustrating having to wait for tests that will probably pass.

kaspnilsson• 7 years ago

I used to work on Gerrit at Google, it's open source: https://github.com/GerritCodeReview/gerrit

noncoml• 7 years ago

Hate to be that guy, but why is this even in the front page, let alone #1?

dstick• 7 years ago

Well I guess I’m “that guy” who finds it highly interesting as an outsider with a very high interest in search. Never knew what tools Google’s using (never bothered to look it up either) and this list is a feast of “Aha!”’s for me :)

Hopefully that answers your question ;-) Same as with a discussion about patents the other day. It’s fascinating to learn / read / understand how big businesses work as someone who’s never worked at one nor aspires to work at one, but does want to build one ;-)

tomcam• 7 years ago

Just winging it here, but maybe because these are some of the most influential companies on the face of the Earth, they employ tens of thousands of HN users, they are the envy of maybe half of all employees on the planet, their programming systems effect everyone reading HN, and many people here are genuinely interested in what happens at ginormous corporations that employ other hackers.

jaytaylor3• 7 years ago

Is there something comparable to the web-based IDE called Cider?

Now that Amazon took cloud9 (c9.io) out of the market, it seems like there is a sizeable void.

I want web-based Spacemacs or Evil! Or even just intelli-j.

kajecounterhack• 7 years ago

Cloud9 is still there https://aws.amazon.com/cloud9/?origin=c9io

jaytaylor3• 7 years ago

Now with that AWS lock-in, no thanks. It only serves as a reminder of how early adopters get burned, similar though not as bad as the whole FoundationDB->Apple fiasco; that one affected me personally and drove home this lesson.

I don't enjoy being the insect who claimed a tall blade of grass only to be mowed down first. If you value your time and value stability, don't use closed-core tools.

tsechin• 7 years ago

I've tried Eclipse Che (listed in the article), and it's pretty good. I'm looking forward to Che switching its editor to Eclipse Theia, which promises to be "the VS Code experience in the browser".

anton_kosyakov• 7 years ago

You should check out Gitpod (https://www.gitpod.io). It was launched recently and provides disposable, ready-to-code development environments for GitHub projects: https://dev.to/svenefftinge/introducing-gitpod-frictionless-...

IDE is based on Theia (https://theia-ide.org), i.e. VS Code with cloud-first mindset.

joshstrange• 7 years ago

Hmm, I was looking at Gitpod's pricing page and I do not at all like that clicking on the "Gitpod for teams" link takes me immediately to github to auth my account, uh no thanks.

jankeromnes2• 7 years ago

I'm totally biased, but I think Gitpod might be the closest thing to Cider here, being a hosted IDE + container solution that continuously prebuilds your workspaces along with your code.

The other options are just IDEs in a browser where you still have to hammer together your own setup + build manually every time.

HuggableSquare• 7 years ago

I recently discovered Coder's open source version of their web-based IDE (https://github.com/codercom/code-server). It's pretty much VS Code in the browser, which is almost exactly what I was looking for. You just download the binary, point it to a folder and it works exactly the way you would expect (including the terminal running under the account the server runs as so you get to keep your shell of choice and all that fun stuff).

mherdeg• 7 years ago

I wonder how many of the we-left-Facebook-and-miss-it sorts of services don't have an obvious Google corollary (I'm thinking of Scuba/Honeycomb)

simula672• 7 years ago

Is there an alternative to BigTable which written in C or Go ?

stingraycharles• 7 years ago

http://www.hypertable.org/ comes to mind. It's a BigTable implementation written in C++, it's fairly mature as well.

simula672• 7 years ago

Thanks!

puzzle8• 7 years ago

If C++ is an option, Baidu has a clone called Tera.

q3k4• 7 years ago

ScyllaDB, which is basically a C++ Cassandra reimplementation, which in turn borrows a lot of design ideas from BT.

dsl2• 7 years ago

I really wish they would add memegen to G Suite.

alpb2• 7 years ago

Dory is probably more realistic.

dsl2• 7 years ago

Dory is available now! If you go into presenter view in slides there is an option to launch it. It will put a short URL at the top of your presentation.

antimora• 7 years ago

Anything for Amazon tooling?

For example, I see Pipelines could be replaced by CodeDeploy. I hear nix is similar to the brazil build system. Any other similar mapping by others?

mdeeks• 7 years ago

What does Google (or others) use for an intranet or team documentation? We're using Quip and it is just a black hole where documents go to die, never be seen again no matter how badly you want to find it.

bauerd2• 7 years ago

Can't tell for Google, but Confluence is widely adopted (still I love to hate on it)

Cthulhu_4• 7 years ago

The challenge with Confluence - and all documentation systems for that matter - is that it needs a manager, someone who makes sure it's maintained, old / outdated pages get removed, the landing page is neat, and authors are encouraged to keep it updated.

I can imagine that can quickly become a fulltime job though. Still, a formal role of documenter / archivist in a bigger software endeavour should be considered.

verst• 7 years ago

It used to be Google Sites (then part of Google Apps for Business which is now GSuite), Docs and Slides which were indexed by an internal Google search engine called MOMA (which I believe was powered by the Google Search Appliance, but my recollection on this last point is faint)

bitskits• 7 years ago

Within Google there are several tools, but the most common (and IMO best) is markdown. (Docs is a close second.)

Pros: Get's checked in like code, gets reviewed, and leaves a nice paper trial of edits.

Cons: Doesn't invite comments/discussions as well as Docs does.

puzzle8• 7 years ago

For many years there was also a wiki that just didn't refuse to die. I think that when I left there was yet another attempt going on to migrate the content away from it. Surprisingly, one of the main appeals was that its availability was independent of Google's production environment (Borg, GFS and co.). If Docs or Sites were down, which was slightly more likely inside Google because most employees were on dogfood builds, it was going to be a fun day...

alexhutcheson2• 7 years ago

Amazon had a giant internal wiki that they used for team information, playbooks, design docs, etc. Formatting was kind of ugly, but the barrier to making an edit was super low, and having everything in one searchable place was super useful. From experience, I think any system that requires more than a few minutes to make an edit ends up not being updated and going stale pretty quickly.

Source: https://www.xwiki.com/en/references/amazon

malkia• 7 years ago

maybe piper|citc -> VFSForGit

smueller12342• 7 years ago

This is pretty misleading: a lot of the presented open source equivalents are nothing like the original. For example, comparing HDFS to Colossus is... optimistic? Having used / worked on/with both, the similarity is roughly that they both store bytes. This would be a lot less misleading if there were annotations and if the "open source/real world" didn't mix together other large companies' products with open source. For example protobuf is open source, but this makes it look like it isn't.

puzzle8• 7 years ago

Yeah, HDFS is a toy in comparison. It's barely comparable to the original GFS, which was retired... Almost a decade ago?

alexhutcheson2• 7 years ago

If you needed something Colossus-like in the open source world, what would you use? Is there anything available that's better than HDFS for the "general-purpose, big, distributed file system" use-case?

puzzle8• 7 years ago

Ceph?

q3k4• 7 years ago

Ceph. Although it's slightly upside down with it's block / objects semantics (and as such, you don't exactly get the namespaced append-only semantics of Colossus), and doesn't depend on an excellent separate BigTable-like and Chubby-like layer underneath, instead rolling their own.

romka24• 7 years ago

There is no listed equivalent of RecordIO. What do people use for high-reliability journals?

When I needed something like RecordIO to store market data, I couldn't find anything. So I implemented https://github.com/romkatv/ChunkIO. I later learned of https://github.com/google/riegeli (work in progress), which could've saved me a lot of time if only I found it earlier. I think my ChunkIO is a better though.

mempko• 7 years ago

Would Riak's Bitcask format fit the bill here?

https://riak.com/assets/bitcask-intro.pdf

romka24• 7 years ago

This format looks somewhat underpowered. If one record is corrupted, there is no way to read anything after it. For the same reason there is no lookup/sharding support, such as finding the first record that starts in the second half of the file. If a writer crashes, a new instance of writer cannot append to an existing file without reading its whole content and truncating on the last readable record.

shereadsthenews6• 7 years ago

That appears to be exactly RecordIO. It even has transposition. Is there a reason that doesn't meet your requirements?

Edit: it even includes an open source implementation of Cord, which they've renamed to Chain for some reason.

romka24• 7 years ago

That appears to be exactly RecordIO.

I suppose you mean "exactly" in a figurative way. Riegeli is definitely inspired by RecordIO and is meant as a successor to it but it's not RecordIO.

Is there a reason that doesn't meet your requirements?

I need to store timeseries with fast lookup by timestamp. Riegeli doesn't support this out of the box. If I had discovered it before I built ChunkIO, I probably would've pulled the low-level code out of it and added timeseries support on top. Or maybe not. Reliability is very important to me and it's risky to use work-in-progress software that may or may not have any production footprint (I'm no longer with Google so I don't know if they use it internally.)

Xg2xg: Lookup table of similar tech and services for ex-Googlers

Related Stories