abadid

The hype doesn't always live up to reality.

More proof that it is really slow to access data in a database over JDBC.

Short summary: Trino uses JDBC in ways in which it was not designed and this can result in massive performance bottlenecks. Sometimes partitioning source data can mitigate the bottleneck.

A lot of people don't realize how slow it is to access data in bulk from a database using JDBC. JDBC was never designed for the use case that Trino uses it for.

This article summarizes research from my lab in collaboration with ByteDance published in CIDR (a computer science conference held in Amsterdam two weeks from now) on a new columnar format designed for ML workloads.

Why use PostgreSQL instead of columnar databases that are likely to perform way better for these types of analytical workloads?

I agree that in theory they could both co-exist for the reasons you state, but in practice I think it's unlikely a company that invests in a data fabric (which is largely a technology cost) is going to simultaneously invest in the incentives for the data product creators that are necessary for the data mesh not to become the wild west.

In general, whenever you need to perform a join (of multiple datasets), that ends the pipeline of local operations on a partition. Other operators as well that necessarily require data from other partitions end local pipelines. This is why linear scalability is not completely achieved in practice. Most interactions with data cannot be performed in a completely partitionable way.

Usually those other operations which force the local pipeline to end occur in a query plan prior to hitting any kind of tradeoff of doing too much in a local pipeline, since local pipelines are SO much faster than what happens when communication is required.

I'm the author of this piece. I'm happy to respond to comments in this thread.

IMO, it's hard to put a price on strong isolation and consistency. Being able to write an app that that uses atomic transactions, that are isolated from concurrently running transactions, and that see the correct data is something that translates to reduced programmer time and effort, and improves user experience. Many programmers discount those important features when they start out, but they'd be better served including them in the price comparisons of different products that are out there.

See what I wrote below regarding Spanner. The same thing applies to the CockroachDB solution. If you run 2PC for multi-region transactions that is very slow (increases latency), and prevents conflicting transactions from running for longer periods of time (decreases throughput).

Yes --- the post explicitly states that deterministic execution is a prerequisite.

SLOG is CP from CAP, so indeed suffers from unavailability in the event of a network partition.

Let's say you have a transaction that writes data located in different quorums atomically. This is called a "multi-home" transaction in SLOG. Most of the machinery in the paper is dealing with that case. SLOG can do that with half-round trip latency. No idea how long it would take the version of Spanner you described to do that, but presumably at least two full round trips across quorums for two-phase commit.

Latency is measured from the client. In the example in the post (and more details in the paper), you see the latency tail from when clients access data that is far from them. The challenge is to make multi-home transactions no worse than regular Paxos latency. In previous systems, this required multiple rounds of communication across the homes that are involved in that transaction. In systems like PNUTS, they would disable such transactions altogether. SLOG's ability to handle such transactions with latency no worse than Paxos is a big step forward.

As mentioned in the post: "There are several ways to achieve [serializability] — such as via locking, validation, or multi-versioning."

Deadlock happens under some, but not all implementations of serializability via locking. There have been several database systems developed in my lab that use locking to achieve serializability, but yet never deadlock. Examples include:

(1) Calvin: http://www.cs.umd.edu/~abadi/papers/calvin-sigmod12.pdf (2) Orthrus: http://www.cs.umd.edu/~abadi/papers/orthrus-sigmod16.pdf (3) PWV: http://www.cs.umd.edu/~abadi/papers/early-write-visibility.p...

Bottom line: serializability does not necessarily mean deadlock. Deadlock can be avoided via non-locking implementations, or even in well-designed locking implementations.

One of the points in the conclusion of the post warrants being reiterated at this point:

"If you find that the cost of serializable isolation in your system is prohibitive, you should probably consider using a different database system earlier than you consider settling for a reduced isolation level."

Glad to see to see this post on HN. I'm the author and happy to respond to questions in this thread.

Please see: http://www.cs.umd.edu/~abadi/papers/determinism-vldb10.pdf

magicalhippo asked a similar question both with regard to complexity and garbage collection. I think it is best to combine these threads, so please see my response there ... https://news.ycombinator.com/item?id=19003212

Every transaction has an numeric identifier. These identifiers are used to order versions of updates to a data item. Versions from higher transaction IDs are considered to be "after" lower ones. Reading versions as of a particular version identifier must be consistent. But the system as a whole needs not be deterministic.

It is one coordinator per transaction. As far as coordinator robustness, I discussed this in the following paragraph from my post:

"There are two categories of work-arounds to the blocking problem. The first category of work-around modifies the core protocol in order to eliminate the blocking problem. Unfortunately, these modifications reduce the performance --- typically by adding an extra round of communication --- and thus are rarely used in practice. The second category keeps the protocol in tact but reduces the probability of the types of coordinator failure than can lead to the blocking program --- for example, by running 2PC over replica consensus protocols and ensuring that important state for the protocol is replicated at all times."

The advantage of removing 2PC is achieved in Calvin and Fauna. But I'm arguing in this post that it can also be achieved in nondeterministic systems (or really any system) and maintain the guarantees of that system.

Actually the approach described by this post does much better than 2PC under contended workloads because it removes the cloggage problem (see the section entitled "The problems with 2PC").

(1) I'm not sure I fully understand what you are referring to wrt the "acks" but either way, one major difference is that the acks don't have to be made durable in the alg. described by the post. Also, the other key thing to note is that alg. described by the post does not suffer from the blocking or cloggage problems of 2PC.

(2) The post was already too long, so I didn't cover all the cases. The code rewrite algorithm described in the post is indeed O(n^2) if every shard has the possibility of a state-based abort. However, the number of shards that have possibilities of state-based aborts is known before the transaction begins, and since the same values are being sent everywhere, you can always use standard network broadcast techniques to bring the complexity back to O(n).

(3) Garbage collection can work via a high water mark. You keep track of the highest transaction number for which all transactions less than this number have been completed, and can garbage collect values for those versions.

What you say is correct :)

I think I misunderstood what you were saying. By "available" I mean that the version exists at the time of the read. As far as available wrt failure, this is no different than any other system. If you don't synchronously replicate data, you can cannot process transactions over data that is offline (and still guarantee ACID). The transaction cannot proceed whether or not you are using 2PC.

2PC requires two network round trips and a global synchronization point. Plus it has the blocking problem and the cloggage problem described in the blog post. The first read of OLLP is before the first lock is acquired, so it doesn't result in any additional cloggage, and definitely no blocking problem.

OLLP by itself is not guaranteed to make progress, but it is easy to prevent starvation via exerting more control at the preprocessing layer. The three papers to read related to this subject are:

(1) http://www.cs.umd.edu/~abadi/papers/determinism-vldb10.pdf

(2) http://www.cs.umd.edu/~abadi/papers/calvin-tods14.pdf

(3) http://www.cs.umd.edu/~abadi/papers/determinism-eval.pdf

Aborts due to OLLP are state-based aborts, so other shards fail via the conditional logic described in the blog post.

This requires a longer response that I have time write now. Please check back tomorrow night ...

Unfortunately 3PC exacerbates the performance problems of 2PC.

Yes, but it is possible to architect the system such that Y_i is available.