vvern

How are folks doing CI on tangled?

Now I want to see benchmarks

Notably missing both Go and Rust

Why does clean room even matter given SQLite is in the public domain?

Still, I think it’s the right default to start with serializable. Then when you have performance issues you can think long and hard about whether relaxed isolation levels will work in a big free way. Better to start with a correct application.

About time for the 6th Edition, eh? What would folks include in it?

- Vector databases and hybrid search?

- Object storage for all the things? Lake houses. Parquet and beyond.

- Continuously materialized views? I'm not sure this one has made the splash but I think about Naiad (Materialize) and Noria (Readyset)

- NewSQL went mostly mainstream (Spanner wasn't included in the last one, but there's been more here with things like CockroachDB, TiDB, etc)

This is nice double entendre because tonic is a pleasant gRPC server library in rust!

https://github.com/hyperium/tonic

It should be this way. Clients should have some protocol to communicate the schema they expect to the database probably with some versioning scheme. The database should be able to serve multiple mutually compatible views over the schema (stay robust to column renames for example). The database should manage and prevent the destruction of in use views of that schema. After an old view has been made incompatible, old clients needing that view should be locked out.

You’re right that it would run on a block chain, but that fact would primarily exist to power some marketing. Everybody would end up interacting with it through a single centralized web site and API because it’s the only usable way to get it to work.

Thank you for writing this. This comes up constantly, and it'll be great to have another reference to cite.

Another interesting thing about TPC-C is how the cross-warehouse contention was designed. About 10% of new order transactions need to do a cross-warehouse transaction. If you can keep up with the workload, then the rate of contention is relatively low; most of the workload isn't pushing on concurrency control. If, however, you fall behind, and transactions start to take too long, then the contention will pile up.

When you run without the keying time, it turns out that concurrency control begins to dominate. For distributed databases, concurrency control and deadlock detection is fundamentally more expensive than it can be for single-node databases -- so it makes sense that a classically single-node database would absolutely trounce distributed databases. I like to think of tpcc "nowait" as really a benchmark of concurrency control because, due (I believe) to Amdahl's law the majority of its execution time ends up just in the contended portion of the workload.

Also very interesting that, as Justin points out, the workload sets up the warehouses so there is never cross-node contention. That's wild! I'm glad they didn't go and benchmark against even more distributed databases (like YugabyteDB, Spanner, or CockroachDB) and call it a fair fight.

Folks, for the love of god, please please stop running TPC-C without the “keying time” and calling it “the industry-standard TPCC benchmark”.

I understand there are practical reasons why you might want to just choose a concurrency and let it rip at a fixed warehouse size and say, “I ran TPC-C”, but you didn’t!

TPC-C when run properly is effectively an open-loop benchmark that scales where the load scales with the dataset size by having a fixed number of workers per warehouse (2?) that each issue transactions at some rate. It’s designed to have a low level of builtin contention that occurs based on the frequency of cross warehouse transactions, I don’t remember the exact rate but I think it’s something like 10%.

The benchmark has an interesting property that if the system can keep up with the transaction load by processing transactions quickly, it remains a low contention workload but if it falls behind and transactions start to pile up, then the number of contending transactions in flight will increase. This leads to non-linear degradation mode even beyond what normally happens with an open loop benchmark — you hit some limit and the performance falls off a cliff because now you have to do even more work than just catching up on the query backlog.

When you run without think time, you make the benchmark closed loop. Also, because you’re varying the number of workers without changing the dataset size (because you have to vary something to make your pretty charts), you’re changing the rate at which any given transaction is going to be on the same warehouse. So, you’ve got more contending transactions generally, but worse than that, because of Amdahl’s law, the uncontended transactions will fly through, so most of the time for most workers will be spend sitting waiting on contended keys.

Some time ago I worked on cockroachdb and I was working on implementing planning for complex online schema changes.

We really wanted a model that could convincingly handle and reasonably schedule arbitrary combinations of schema change statements that are valid in Postgres. Unlike mysql postgres offers transactional schema changes. Unlike Postgres, cockroach strives to implement online schema changes in a protocol inspired by f1 [0]. Also, you want to make sure you can safely roll back (until you’ve reached the point where you know it can’t fail, then only metadata updates are allowed).

The model we came up with was to decompose all things that can possibly change into “elements” [1] and each element had a schedule of state transitions that move the element through a sequence of states from public to absent or vice versa [2]. Each state transitions has operations [3].

Anyway, you end up wanting to define rules that say that certain element states have to be entered before other if the elements are related in some way. Or perhaps some transitions should happen at the same time. To express these rules I created a little datalog-like framework I called rel [4]. This lets you embed in go a rules engine that then you can add indexes to so that you can have sufficiently efficient implementation and know that all your lookups are indexed statically. You write the rules in Go [5]. To be honest it could be more ergonomic.

The rules are written in Go but for testing and visibility they produce a datomic-inspired format [6]. There’s a lot of rules now!

The internal implementation isn’t too far off from the search implementation presented here [7]. Here’s unify [8]. The thing has some indexes and index selection for acceleration. It also has inverted indexes for set containment queries.

It was fun to make a little embedded logic language and to have had a reason to!

0: https://static.googleusercontent.com/media/research.google.c... 1: https://github.com/cockroachdb/cockroach/blob/f48b3438a296aa... 2: https://github.com/cockroachdb/cockroach/blob/f48b3438a296aa... 3: https://github.com/cockroachdb/cockroach/blob/f48b3438a296aa... 4: https://github.com/cockroachdb/cockroach/blob/f48b3438a296aa... 5: https://github.com/cockroachdb/cockroach/blob/f48b3438a296aa... 6: https://github.com/cockroachdb/cockroach/blob/master/pkg/sql... 7: https://github.com/cockroachdb/cockroach/blob/f48b3438a296aa... 8: https://github.com/cockroachdb/cockroach/blob/f48b3438a296aa...

While that’s sort of true, there’s a lot of language specific things that go into making the UX of a debugger pleasant (think container abstractions, coroutines, vtables and interfaces). Specifically async rust and Tokio gets pretty interesting for a debugger to deal with.

Also, there’s usually some language (and compiler) specific garbage that makes the dwarf hard to use and requires special treatment.

How would you compare that to, say, go? I think the unit of distribution in go is a module, and the unit of compilation is a package. That being said, by using `internal` packages and interfaces you can similarly create the same sort of opaque encapsulation.

My biggest issue with rust after two years is just as you highlight: the mod/crate divide is bad!

I want it to be easier to have more crates. The overhead of converting a module tree into a new crate is high. Modules get to have hierarchy, but crates end up being flat. Some of this is a direct result of the flat crate namespace.

A lot of the toil ends up coming from the need to muck with toml files and the fact that rust-analyzer can’t do it for me. I want to have refactoring tools to turn module trees into crates easily.

I feel like when I want to do that, I have to play this game of copying files then playing whack-a-mole until I get all the dependencies right. I wish dependencies were expressed in the code files themselves a la go. I think go did a really nice job with the packaging and dependency structure. It’s what I miss most.

Literally all three of their required features which were not “standard” for swiss tables were requirements for the go implementation. See https://go.dev/blog/swisstable for a nice post on that project.

I’ll nitpick you back: if done correctly, consensus can have quite positive scaling consensus groups can have quite a positive impact on tail latency. As the membership size gets bigger, the expectation on the tail latency of the committing quorum goes down assuming independence and any sort of fat tailed distribution for individual participants.

Cockroach really doesn’t have limitations when it comes to queries. It is kinda magic in that regard. The single partition queries are the special case and are treated as rare. Cockroach ranges are quite small compared to other systems (~256MiB).

Check out https://sqlsync.dev/

It’s all open source, sorry there was no link!

https://github.com/ajwerner/voronoi

I made an implementation in clojurescript that animates the algorithm as it goes a while back: https://voronoi.ajwerner.net/#/app-diagrams

It’s a very beautiful algorithm.

However, after that project I sort of came to dislike Fortune’s algorithm because it isn’t numerically stable with floating point numbers. If you have points that are colinear, or nearly colinear in fp, things can break. The delaunator is better in this regard iirc: https://github.com/mapbox/delaunator

I don’t think in NYC it’s fair to argue that there’s relatively little interest in making transit something people want to use. We can debate about whether the efforts are effective, and certainly many aren’t. However there are vast sums invested in the MTA and a great many folks at the MTA who try to make it pleasant and safe. Additionally there have been added police and military presence in subway stations around the city for months (again, no comment on efficacy). All I’ll say is that there is a ton of interest from leadership on down in making the subway and buses work well for normal people and far more money then congestion pricing cost to implement or will bring in.

This feels like a crazy take. The stronger labor laws part I hear, but the anti-urbanism part I don’t.

Do you have data or an ideology or something? Seems like you’ve been radicalized to a point of view, but by what?

Have you taken a peek at sqlsync? It's a cool take where you write your entire application logic to operate as transactions and queries over a sqlite database but you do it in wasm so that the same transaction logic can run everywhere. The end result is nifty. That's at least my recollection of the thing.

https://sqlsync.dev/

If that’s what you want, try cockroachdb serverless. Scales from zero in well under 1s even for multiregion. Also actually has quite good Postgres support these days.

One approach to mitigate the connection problems for tpcc would be to utilize a connection pooler like pgbouncer or yandex/odyssey. It’s certainly more complexity.

Another suite to look at is sysbench. It’s very flexible, for better or for worse, but it can allow you to create an interesting mix of queries at different scale factors. For something like this where you’re going head to head with Postgres, having more dimensions with more benchmarks isn’t going to hurt. Ideally you’ll see a nice win across the board and get an understanding of the shape of differences.

Can people please, for the love of god, stop running tpcc with think time disabled. When run in this way it is not the TPC-C benchmark, and is not "simulating real database workloads that is considered a modern standard in database applications." TPC-C generally has an open-loop traffic arrival rate that scales with the size of the data and is lightly contended. When run without think time, it becomes closed loop, and generally dominated by the contention that was not supposed to be dominant.

This instance is less bad than some in that it's at least comparing the same sort of database and doing it using the same driver -- so it is at least an apples to apples measurement of something.

Still, please, as a community we need to stop getting rid of the think time and quoting the output as tpmC or as a standard benchmark.

See https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-c... for the spec.

What I’m hungry for is not so much this coherent application for exploring json data but rather a set of React components to integrate explorations like this into a web app.

Does anybody know of any good components or libraries for that?

Transactional workloads over datasets in the single digit petabytes.

Cockroach is also doing 3x replication of the data, so I don’t think that’s particularly relevant here. Cockroach serverless will dynamically scale up sql serving processes based on load. The storage and compute are separated in the cockroach architecture. My point is that if your query load is relatively low, cockroach serverless is definitely cheaper because the storage costs dominate. I think there’s ambiguity on which product is cheaper for a real-world application with meaningful load and data size.

I remain curious about the perception that cockroach is a meaningfully more expensive product. Where does that idea come from?