hugofirth

What specific example or point from the blog post do you disagree with?

I wish Swift wasn’t so tied to the Apple ecosystem, as it has (IMO) a good mix of these

This is not the title of the article, kinda biases the reader to the outcome of the debate don’t you think?

You may be interested in https://www.gqlstandards.org/ and https://www.iso.org/standard/79473.html

TL;DR the ISO standards committees behind SQL are working on bring graph query languages and SQL databases closer together.

Obviously there is work to be done at the storage/query planning layer after that, but I’m hopeful once the surface exists more widely that will drive more work in those areas.

Man I was frustrated by this article. Despite it being on a subject matter I normally ignore, it lured me in with the promise to “dig into data publicly in a way that (so far) I haven’t seen anyone else do”.

I was expecting some novel analysis of in depth data scraped from GitHub APIs, or the results of a qualitative survey of big Java shops, or … something?

Instead what we got was a reposting of a few surface level stats pulled from other blog posts which have tolled the death of Java (or C++ or whatever) ad nauseum.

It’s as if the author started with a narrative and went searching for the line plots to justify it.

To be clear I don’t doubt the growth of Java amongst new learners is slowing down. Partly perhaps due to an image problem relative to other new languages, but likely also due to having somewhat saturated its “total addressable market” and not repositioning itself in new “markets” in the way that, say python has done.

But it’s still a great language for many categories of problem (e.g. high throughput live service) and has rich ecosystem. Build what you like with whatever tool is good for the job. Stop using bad stats to motivate your recent learning choices and just go build stuff :-)

As a data point, this change will mean a lot of work for us (we built cluster discovery on top of Akka Cluster at Neo4j). I'm not thrilled, but I respect their right to make these changes, and personally think that BSL with a timed revert to Apache 2.0 is a meaningful compromise.

Enterprises have to be sustainable. If you're furious with Lightbend for making this change to protect their bottom line, then you can always fork one of the last open source versions and maintain/fix that yourself. But most people won't do that, because if they did they'd likely have been active contributors to the project before now, reducing the ongoing maintenance burden on Lightbend and potentially avoiding them feeling the need to close-source in the first place.

Yes they do

If you're interested in building distributed systems, then I thoroughly recommend Marc Brooker's blog: https://brooker.co.za/blog/

It does though, the primary thesis of the article is that the question (whether or not software engineering is engineering) is important because if software engineering is the real deal, then we're ignoring useful lessons from all the other disciplines.

This is daft. CockroachDB exhibits extremely strong isolation and consistency levels. I believe it is strictly serializable under most circumstances?

Also - anyone who says that x database must be rewritten “because GC” is just making an incredibly un nuanced argument about a nuanced problem. People have built production ready databases in both Java and Go. If you care about low/predictable tail latencies then you have a bunch of other more important problems to solve before you worry about the behaviour of a modern garbage collector. For example: how good is your cache hit ratio? How are your synchronous replication protocols affected by grey failures? That kind of thing.

Another thing which makes the Raft/Paxos vs new-consensus-algorithm comparisons complicated is caching.

If your raft state machines are doing IO via some write through cache (which they often are) then having specific machines do specific jobs can increase the cache quality. I.e. your leader node can have a better cache for your write workload, whilst your follower nodes can have better caches for your read workload.

This may lead to higher throughput (yay) but then also leave you vulnerable to significant slow-downs after leader elections (boo).

What makes sense will depend on your use case, but I personally agree with the author that multiple simple raft/paxos groups scheduled across nodes by some workload aware component might be the best of both worlds.

Enterprise licenses for on premise Neo4j are certainly for a specific customers with specific needs, but there is always the DBaaS (https://neo4j.com/cloud/aura/).

Or if you absolutely need on premise and are small there is the startup program for free enterprise licenses (https://neo4j.com/startups/)

Competition is good, and from my experience with both Nomad feels like a more complete product than kubernetes

I couldn't agree more about curiosity mode (I'm going to use that phrase liberally). Despite reading papers for many years, I rarely go in cold. I browse blog posts and twitter, ask myself a series of questions, then try to find/read papers to answer them. Of course this only leads to more questions, and so the journey continues.

I also agree with the recommendations for "Designing Data-Intensive Applications" and "Database Internals". Though, having read the latter for a book club at $employer, I felt it served better as a sort of "index for the space" for people who already had some DB experience, rather a true introduction.

I have many recommendations of different kinds:

## Blogs:

- http://muratbuffalo.blogspot.com/

- https://bartoszsypytkowski.com/

- https://decentralizedthoughts.github.io/

- https://www.the-paper-trail.org/

- https://blog.acolyer.org/

- https://pathelland.substack.com/

## Other web resources

- https://aws.amazon.com/builders-library/ - set of resources from Amazon about building distributed systems

- https://www.youtube.com/playlist?list=PLeKd45zvjcDFUEv_ohr_H... - lecture series from Cambridge

## Books

- https://www.cl.cam.ac.uk/teaching/1213/PrincComm/mfcn.pdf - A great book on the maths of networking (probability, queuing theory etc...)

I believe it would have alleviated some of the issues yes. In general I’m excited for the benefits “pauseless” GC can bring to soft real-time systems on the JVM. Unfortunately, for now we have to continue supporting G1 and friends.

We're a big Java/Scala shop already, so Akka was easier to integrate :)

To offer a slightly dissenting opinion, we’ve had many issues with Akka over the years:

- if you roll your cluster membership a lot the dotted version vectors which are created by Akka distributed data grow unbounded. Eventually they will start making gossip messages exceed the default maximum size (a few kB IIRC) and fail to send.

- in the presence of heavy GC Akka cluster has a really bad time. Members will flip flop in marking each other unavailable. Eventually this will render the leader unable to perform its duties and you will struggle to (for example) allow a previously downed member to rejoin the cluster.

- orderly actor system shutdown will also fail under high GC, which is problematic as sometimes you need to restart your actor system.

- split-brain resolution is really really hard to get right. The Akka team have recently made theirs open source I believe which is good, but back when we were building with Akka cluster it required a Lightbend subscription.

- If you aren’t all in on Actors, the integration point between Akka and the rest of your codebase can be a little odd. You often feel like you should reach for `Patterns.ask` (a way of sending a message to an actor and then getting a Future back which will complete on a particular response) but then people tell you that’s an Anti pattern.

————

Having said all the above, if you’re able to go all in on the Actor pattern and you’re unlikely to hit high GC then you should give Akka cluster a try. The problems it tackles are genuinely hard and you should build on their hard work if you can. In particular they offer (in distributed-data) the most robust/complete set of CRDTs I’ve yet come across. Many other CRDT libraries expect you to bring your own gossip protocol and transport layer.

I never know where I sit on stuff like this. On the one hand if you’re confluent I think it makes total sense to own this part of your infrastructure. Especially if it lets you improve your operability story.

On the other hand I feel like projects should try and use open source “building-blocks”, like etcd and zookeeper, when building their distributed systems. Not only does this help iron out correctness bugs, but it also means that more people are familiar with the quirks, limitations, requirements etc.... of these tools. For example, I think I would be frustrated to hear that K8s were implementing their own raft.

I've said this before on other threads but frankly, scientific publishers represent institutionalised theft of tax payer money:

- Academics (most often publicly funded via grants and university salaries) do the work for free.

- They are expected to learn to use LaTeX and to typeset their work for free.

- They are expected to copy-edit the papers for free, or else pay a copy editor themselves with, you guessed it, public funds.

- Volunteer Academics (on university time and therefore, again, public money) are expected to review the work for technical accuracy and novelty. If done well this is extremely time consuming.

- Finally, the Journals have the temerity to charge the same universities who produce their product millions of pounds a year in journal subscriptions and Open Access fees.

- Finally finally, none of the Authors are ever paid for their work. Not that it matters, because again: public funding should mean public access.

The most frustrating part is that Academics themselves are locked into this system by the career prospects conferred by prestigious journals/conferences.

I like to hope the ACM and other signatories will face a backlash for this. But they most likely wont

Frankly, scientific publishers represent institutionalised theft of tax payer money:

- Academics (most often publicly funded via grants and university salaries) do the work for free.

- They are expected to learn to use LaTeX and to typeset their work for free.

- They are expected to copy-edit the papers for free, or else pay a copy editor themselves with, you guessed it, public funds.

- Volunteer Academics (on university time and therefore, again, public money) are expected to review the work for technical accuracy and novelty. If done well this is extremely time consuming.

- Finally, the Journals have the temerity to charge the same universities who produce their product millions of pounds a year in journal subscriptions and Open Access fees.

- Finally finally, none of the Authors are ever paid for their work. Not that it matters, because again: public funding should mean public access.

The most frustrating part is that Academics themselves are locked into this system by the career prospects conferred by prestigious journals/conferences.

I’m not normally one for beating the “nationalise them” drum, but if there has ever been a case for businesses to be dismantled and put in public hands it’s these parasites.

Sincerely, a Scientist :-)

I don't like Oracle any more than the next person, but this just isn't true at all. Oracle is a robust product which fills an important part of any technical stack. Building and supporting databases is hard.

Presumably this structure means the company pays the tax to the government, and are, in essence, buying back their own shares :)

There are plenty of industries where companies don't exist in a particularly free market, and have to operate according to extremely strict regulation. Thats all I meant :)

This might be stupid - but why are these credit ratings agencies allowed to exist in a comparatively free market? Surely a function like the one they provide is better handled by a Not-For-Profit governmental body?

Without this - or incredibly tightly policed regulation of the existing private agencies, I don't see how an outcome like the one described here isn't mathematically guaranteed?

For a C codebase, Postgres[1] wins for me hands down. It's clean and suuuuuuper well commente, such that with a little context you can dive into something very complex and still get a feel for what is going on.

[1] https://github.com/postgres/postgres

Using Neo4j community edition would certainly make your life easier in terms of the queries you might want to express, but for this amount of data you can get away with postgres, as you observe.

If you want a simple in memory graph modelling library then check out Apache Tinkerpop. Its great.

Culture fans: These are not the knife missiles you were thinking about.

Though I'm not sure the US having access to those would make me feel any better :)

I suspect you will thoroughly enjoy this twitter account: https://twitter.com/cultureshipname?lang=en

True. Though in tradeoff it has massively increased throughput. I'm always frustrated when I see blockchains recommended as a solution for a consensus problem in anything but a zero-trust scenario