noahl

There are some semantic differences compared to POSIX filesystems. A couple big ones:

  - You can only append to an object, and each object can only have one writer at the time. This is useful for distributed systems - you could have one process adding records to the end of a log, and readers pulling new records from the end.
  - It's also possible to "finalize" an object, meaning that it can't be appended to any more.

Slight nit: "zonal" doesn't necessarily mean "not replicated", it means that the replicas could all be within the same zone. That means they can share more points of failure. (I don't know if there's an official definition of zonal.)

NB: I am on the rapid storage team.

There's now another blog post about Rapid storage specifically: https://cloud.google.com/blog/products/storage-data-transfer... . (That wasn't up yet when the original post was made.)

No, not for GCP stuff.

I don't know of a single GCP product that's been shut down, although I could be missing something. But their track record for GCP is, I think, what you would want a cloud provider's record to be.

(I should mention that I work for GCP. But this is just based on my own memory.)

Got it, thank you! CockroachDB is the only one I know offhand that does what you're looking for. Another comment mentioned Vitess, which might also work.

It seems like there are a lot of options for large scale analytics, but I don't know a lot for high throughout geo-redundant transaction processing.

Spanner offers that on GCP, and I believe CockroachDB offers something similar cross-cloud.

Do you have any specific requirements for which cloud provider you use, or any particular interface you really need?

The solution is posting this marketplace on a forum full of people who might like to buy and sell side projects

I mean to be fair Google has also tried very, very hard to improve home internet access for people, to the point of setting up their own ISP and running municipal fiber networks. That's a pretty big try, and I really wish it had taken off beyond the places where Google Fiber operates.

(NB: I work at Google, but this comment has nothing to do with my work.)

Microsoft/windows is hosted on Azure DevOps, and they have also blogged about what they've done to improve its performance!

Here's a recent post: https://devblogs.microsoft.com/devops/introducing-scalar/

Interestingly, there was an ancient Greek / Roman equivalent: amphora (pottery vases) with bumps that fit into standard cargo racks in ships.

The first picture in the Wikipedia article is what I mean: https://en.m.wikipedia.org/wiki/Amphora .

Someone close to me has Celiac disease, and getting an official diagnosis will let you do more than just eliminate gluten. For one thing, you could participate in one of the multiple ongoing drug trials for Celiac that are looking for participants.

The Celiac foundation has a list of them if you're interested, but you'll need a Celiac diagnosis to be eligible.

I'd be curious to know what the libraries are that allow this sort of database engine design. Are there any open source examples?

Imagine if Barnes and Noble had done it, and you could drive to one of their stores to quietly sip a drink and peruse the new machines they had provisioned recently.

I think Microsoft's strategy (I work there, but I don't have any special knowledge of this) is to be the best possible platform for development in general, not just machine learning. That's why they bought GitHub, and it's also why they have WSL, remote debugging on Linux, etc. I expect they will go wherever they think developers want to be to achieve that goal.

I can see this hurting traditional Linux graphics systems and development tooling, but I think the main dynamic driving this is not proprietary-vs-open, it's Microsoft's ability to coordinate large numbers of developers, because they have a massively more united org than random hobbyists.

If you combined all of the efforts of all of the people working on Linux tools for the past 20 years and got them working on the same thing, it would probably be an amazing stack, and Microsoft would probably just support it. Instead, the open source efforts have been scattered among a lot of different ideas, which means there hasn't been a single windowing toolkit, IDE, or whatever for people to rally around and all contribute to. And as a result, Microsoft can show up, get a few hundred people pulling in the same direction, and have the market-leading product not through any dirty tricks, but just by putting in more coordinated effort than anyone else.

Can you give some examples of the mindset of construction and manufacturing businesses? It would be interesting to me, and I think to other people here too.

I have a suspicion, which I have never verified, that the things that make emacs great are basically the things that made Lisp Machines great back when those were a thing - that emacs is best viewed as a Lisp Machine embedded in a modern operating system.

I never used a real Lisp Machine, but I'd love if someone who did can confirm this.

It was released, and was used for the Windows repo. Search "vfs for git" - it's open source.

Then macOS removed kernel extensions, so they came up with a different approach and released that too. Search "scalar" to see it.

I wonder if anyone uses greenhouses as carbon capture for traditional power plants, rather than running their own mini-power plant mostly for the CO2 output.

I don't work there any more, but back when I was at Google Cloud we developed our own benchmarking suite, partly to answer questions like this. It's open source, you can run it too: https://github.com/GoogleCloudPlatform/PerfKitBenchmarker

If you adopt this article's three-category system (or any system with a finite number of ordered categories), then I think in the long run the number of positive and negative changes have to converge toward equality.

If we hear more stories of negative changes, then maybe the negative ones are more memorable, or people are more likely to comment on them.

I have worked a job that was salaried, in the sense that I got paid a lump sum for the year and didn't track my hours, but also paid extra when I was on call. This is possible, and it works fine.

Making a concept intuitive and easy to understand is extremely difficult to do. A person who could do this would probably have to understand Paxos better than Lamport does.

I wouldn't assume that Leslie Lamport doesn't want his algorithm to be understood. It's just that explaining things is really, really, really hard.

(This is also why teaching is an incredibly difficult profession.)

The trouble is that if you have to understand or correctly guess internal Google politics like this in order to know which products you can really rely on, then it's effectively not reliable.

With that said, I'd feel fine relying on VMs, disks, managed SQL, managed Kubernetes, etc. Maybe I'm too optimistic. And Google is giving exactly the guarantees they promised - 1 year of notice before shutting down a product.

The point is that giving over the names would be bad. What he did is better.

Supporting people's right to self-determination despite negative business consequences is admirable, and I wish we saw more of it.

That sounds like the current use case for helicopters.

Are you suggesting that Google try to poach Satya Nadella? :) I would give a lot to be a fly on the wall for that conversation.

Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible.

The main problem is, I think the author is wrong about what Google's "crown jewel" is. Yes, Google has a huge index, but most queries aren't in the long tail. Indexing the top billion pages or so won't take as long as people think.

The things that Google has that are truly unique are 1) a record of searches and user clicks for the past 20 years and 2) 20 years of experience fighting SEO spam. 1 is especially hard to beat, because that's presumably the data Google uses to optimize the parameters of its search algorithm. 2 seems doable, but would take a giant up-front investment for a new search engine to achieve. Bing had the money and persistence to make that investment, but how many others will?

Out of curiosity, what makes it not the right solution? (I work on the Azure DevOps team.)

No idea, I don't work there any more. It would be really interesting to know though.

This is interesting, but I think the authors don't talk enough about CPU and memory utilization. To me, the "classic" Google distributed systems architecture puts different tasks in different logical servers (doesn't matter if they're separate physical servers or not), which gives those servers more predictable memory and CPU usage, which in turn enables tighter bin-packing of jobs in the datacenter. The price they pay is needing a really, really fast in-datacenter network, but in the past they've been okay with this.

The paper proposes putting application-specific processing and memory caching on the same host, which might give the combined server less consistent CPU usage and therefore lower utilization, but will also eliminate the network hop from application server to in-memory cache. It seems intuitively reasonable to me to give up some CPU utilization in exchange for eliminating an all-to-all network connections stage, but I would like to see a real cost and speed comparison.