mkleczek

Much more principled (and hence less of a foot-gun) way of handling conflicts is making them first class objects in the repository, like https://pijul.org does.

Shameless plug: https://github.com/mkleczek/pgwrh automates it quite a bit.

From my experience the issue really is, unfortunately, that it is impossible to tell if a particular detail is irrelevant until after you have analyzed and answered all of them.

In other words, it all looks easy in hindsight only.

At least in case of the kitchen contractor, you can trust all the electrical equipment, plumbing etc. is going to be connected in such a way that disasters won't happen. And if it is not, at least you can sue the contractor.

The problem with LLMs is that it is not only the "irrelevant details" that are hallucinated. It is also "very relevant details" which either make the whole system inconsistent or full of security vulnerabilities.

I am Polish and I was 15 in 1990 when the Berlin Wall fell.

Lately I was thinking if it was only me or my fellow Poles remembering 90s as times of freedom and hope.

Thanks for confirming it is much wider experience and memory.

Indeed. The organization changed in the passing years.

My worry is that there is more will to just dissolve it instead of working on improving it.

This.

It looks like it goes in cycles: after major catastrophic event subsequent generations that don't remember the catastrophe are willing to engage in another one (I think of WWI and WWII as one event). If the memory was stronger there would be more pressure to find other means of resolving issues.

Eg. as a "half boomer" European I remember well why EU was created. It looks like the reasons have been largely forgotten now.

The economic interest is the US ability to as rapidly as possible convert those shipyards to military shipyards during a large scale prolonged war.

Nah, that doesn't add up. US needs _ships_ and SOTA military equipment to make sure that any military conflict is as short as possible (ie. US wins). Losing money on unused production capability does not make sense because in case of prolonged conflict there is time to build the capability (as it happened during WWII).

In reality, what you call "prolonged military conflict", is nothing more than normal international competition. One could even argue US is in prolonged military conflict since WWII. In which case making rational decisions based on hard economic criteria (ie. not losing money) is the key to success.

Insofar as the country being conquered and Americans being slaughtered wholesale would be against our economic interests lol There are clear national security reasons for the government to prop up shipbuilding and semiconductors.

Are you saying countries without shipbuilding facilities or not producing semicondutors are being conquered and their citizens being slaughtered?

I'd say that is fear mongering done by the people doing business on "national security".

The question is: how do you define "national security" and "other strategic value"? At the end of the day both really mean economic interest. Especially in case of US.

So if someone says "national security" is above economic interest of US, I would say these people mean _their_ economic interest is above economic interest of US and use both terms as a cover.

With current military technology it is not really possible, is it? https://www.youtube.com/watch?v=NdppYYfQJgg describes it really well.

So the question is more about what part of means of defense you outsource. And what parts of means of defense are outsourced by your enemies.

You don't want to base your defense on inferior shipbuilding capabilities, do you?

The theory is that in both cases (ie. with and without tariffs) shipyards are going to die sooner or later. It is better for the society to let them die as soon as possible and direct efforts to things we are better at while taking advantage of cheaper ships produced elsewhere.

This.

In-RDBMS computation specified in declarative language with generic, protocol/technology specific adapters handling communication with external systems.

Treating RDBMS as a computing platform (and not merely as dumb data storage) makes systems simple and robust. Model your input as base relations (normalized to 5NF) and output as views.

Incremental computing engines such as https://github.com/feldera/feldera go even further with base relations not being persistent/stored.

https://news.ycombinator.com/item?id=46432107

I wonder if (when?) elites are going to use and support Bitcoin. Oppressive governments will force citizens - even such powerful as judges - to search for escapes.

Here on HN I will be downvoted to oblivion but well... let's be it:

There is no other way for us mortals than to go back to cash... Or start using Bitcoin. Be your own bank. Vote with your money.

I don’t see how this is true.

There is a Bitcoin seller B, a thieve T and a victim V.

T proposes to buy Bitcoin from B. T offers a new iPhone for a very low price to unsuspecting V. V agrees to buy it. B gives T account details and transaction reference so that T can transfer money to B's account. T gives these details to V. V transfers the money. B transfers Bitcoin to T. T disappears.

If only transaction reference contained information that the transfer is about buying Bitcoin, V would have never paid the money.

The scheme was quite common in UK because banks did not like Bitcoin so Bitcoin sellers and buyers avoided referencing it in bank transfers.

* Based on the data they identify

* easy to remember

(which means human readable and related to the actual information which makes them easier to remember)

These actually are the most important features.

Example: transaction references not related to the actual subject of the transaction (ie. what is being paid for) is enabler for MITM scam schemes.

Short is convenient

Nah. Short is crucial for identifiers to be effective for computers to handle (memory and CPU efficiency). Otherwise we wouldn't need any identifiers and would just pass raw data around.

* versioned - Versioning is only interesting because you’re trying to derive from real data.

Nah. Even UUID formats contain version information.

* easy to index - Sure.

* sortable - Nice to have at best.

These are directly related (and in the context of UUIDv4 vs UUIDv7 discussion sortable is not enough - we also want them to be "close" to each other when generating so that they can be indexed efficiently)

This hacker news article was given a surrogate key, 46272487. From that, you can determine what it links to, the name/date/author of the submission, comments, etc.

Do not encode identifying information in unique identifiers! The entire world of software is built on surrogate keys and they work wonderfully.

The amount of manual work required to manage duplicates is in no small part the result of not thinking enough about the identifiers and simply slapping surrogate keys on the data.

Your notion that you can avoid sharing internal ids is technically true, but that didn’t mean it’s a good idea. You’re trying force a philosophical viewpoint and disregarding practical concerns, many of which people have already pointed out.

What some call "philosophical viewpoint" I call "essential complexity" :)

But to answer your question, yes, your customer will probably have some notion of a transaction id. This is why everyone gives you invoice numbers or order numbers.

We are in agreement here: externally visible identifiers are needed for many reasons (mostly technical). The discussion is not about that though but about what information should be included in these identifiers.

This is why everyone gives you invoice numbers or order numbers.

And there are good reasons why invoice or order numbers are not randomly generated strings but contain information about the invoices and orders they identify.

My claim is that externally visible identifiers should possess a few characteristics:

* should be based on the data they identify (not detached from it)

* should be easy to remember (and that means they should be as short as possible, they should be easy to construct by a human from the data itself - so they cannot be hashes of data)

* should be versioned (ie. they should contain information somehow identifying the actual algorithm used to construct them)

* should be easy to index by database engines (that is highly db implementation dependent unfortunately)

* can be meaningfully sortable (that is not strictly a requirement but nice to have)

Coming up with an identifier having these characteristics is not trivial but is going to pay off in the long run (ie. is essential complexity).

I am not sure you are arguing against my claims or not :)

I am not arguing against surrogate keys in general. They are obviously very useful _internally_ to introduce a level of indirection. But if they are used _internally_ then it doesn't really matter if they are UUIDs or sequence numbers or whatever - it is just an implementation detail.

What I claim is that surrogate keys are problematic as _externally visible_ identifiers.

Okay, then lets do an exercise here. A user gives you a transaction ID, and you have to tell them the date they signed up and the date you first billed them. I think yours is going to be way more complicated.

Mine is just something like:

SELECT user_id FROM transactions WHERE transaction_id=X; SELECT transaction_date FROM transactions WHERE user_id=Y ORDER BY transaction_date ASC LIMIT 1; SELECT signup_date FROM users WHERE user_id=Y;

I think you are missing the actual problem I am talking about: where does the user take the transaction ID from? Do you expect the users to remember all transaction IDs your system ever generated for them? How would they know which transaction ID to ask about? Are they expected to keep some metadata that would allow them to identify transaction IDs? But if there is metadata that enables identification of transaction IDs then why not use it instead of transaction ID in the first place?

The article only discusses reasons why formal verification is needed. It does not provide any information on how would AI solve the fundamental issues making it difficult: https://pron.github.io/posts/correctness-and-complexity

Any piece of information that can be used to retrieve something using this index has to be available "outside" your database - ie. to issue a query "give me piece of information identified by X" you have to know X first. If X is only available in your index then you must have another index to retrieve X based on some externally available piece of information Y. And then X becomes useless as an identifier - it just adds a level of indirection that does not solve any information retrieval problem.

That's my whole point: either X becomes a "real world artifact" or it is useless as identifier.

The UUID would be an example of an external key (for e.g. preventing crawling keys being easy). This article mentions a few reasons why you may later decide there are better external keys.

So we are talking about "external" keys (ie. visible outside the database). We are back to square one: externally visible surrogate keys are problematic because they are detached from real world information they are supposed to identify and hence don't really identify anything (see my example about GDPR).

It does not matter if they are random or not.

How are you going to trace all those records if the requester has changed their name, phone number and email since they signed up if you don't have a surrogate key?

And how does surrogate key help? I don't know the surrogate key that identifies my records in your database. Even if you use them internally it is an implementation detail.

If you keep information about the time information was captured, you can at least ask me "what was your phone number last time we've interacted and when was it?"

I think that spirals into way more complexity than you're thinking.

This complexity is there whether you want it or not and you're not going to eliminate it with surrogate keys. It has to be explicitly taken care of.

DBMSes provide means to tackle this essential complexity: bi-temporal extensions, views, materialized views etc.

Event sourcing is a somewhat convoluted way to attack this problem as well.

Those queries are incredibly simple with surrogate keys: "SELECT * FROM phone_number_changes WHERE user_id = blah".

Sure, but those queries are useless if you just don't know user_id.

"Internal" is a blurry boundary, though

Not for me :)

"Internal" means "not exposed outside the database" (that includes applications and any other external systems)

If you expose the identifier outside the database, it is no longer "internal".

I think artificial and data-less identifiers are the better means of identification that takes into account that things change. They don't have to be the identifier you present to the world, but having them is very useful.

If the only reason you need a surrogate key is to introduce indirection in your internal database design then sequence numbers are enough. There is no need to use UUIDs.

The whole discussion is about externally visible identifiers (ie. identifiers visible to external software, potentially used as a persistent long-term reference to your data).

E.g. phone numbers are semi-common identifiers now, but phone numbers change owners for reasons outside of your control. If you use them as an internal identifier, changing them between accounts gets very messy because now you don't have an identifier for the person who used to have that phone number.

Introducing surrogate keys (regardless of whether UUIDs or anything else) does not solve any problem in reality. When I come to you and say "My name is X, this is my phone number, this is my e-mail, I want my GDPR records deleted", you still need to be able to find all data that is related to me. Surrogate keys don't help here at all. You either have to be able to solve this issue in the database or you need to have an oracle (ie. a person) that must decide ad-hoc what piece of data is identified by the information I provided.

The key issue here is that you try to model identifiable "entities" in your data model, while it is much better to model "captured information".

So in your example there is no "person" identified by "phone number" but rather "at timestamp X we captured information about a person at the time named Y and using phone number Z". Once you start thinking about your database as structured storage of facts that you can use to infer conclusions, there is much less need for surrogate keys.

Using a random UUID as primary key does not mean users have to memorize that UUID. In fact in most cases I don't think there's much reason for it to even be exposed to the user at all.

So what is such an identifier for? Is it only for some technical purposes (like replication etc.)?

Why bother with UUID at all then for internal identifiers? Sequence number should be enough.

Network protocol designers know better and by default embed protocol version number in message format spec.

I guess you can assign 3-4 bits for identifier version number as well.

And yes - for long living data dealing with compatibility issues is inevitable so you have to take that into account from the very beginning.

Preferably, you would design you APIs and services to be idempotent (ie. use PUT not POST etc.)

Using idempotency identifier is the last resort in my book.

That's really an important deficiency of Postgres.

Hash index is ideally suited for UUIDs but for some reason Postgres hash indexes cannot be unique.