ncruces

The article should note this: it's important that you don't go adding other/all little endian platforms to that go:build line.

The article is correct, but this code is only valid for platforms that allow unaligned access.

You can get the full list of platforms that Go considers safe for this from unalignedOK here: https://go.dev/src/cmd/compile/internal/ssa/config.go

You want the intersection of unalignedOK with little endian.

The forum post I linked to has a 3 line example:

  CREATE TABLE t1(a DATE, b JSON);
  PRAGMA strict=ON;
  INSERT INTO t1 VALUES(a,b) VALUES(?1,?2);

Just, you know, explain what should happens when you do this, because the creator of SQLite doesn't know.

When doing so consider the file format, how schema is stored, and what happens when other versions of SQLite touch your file.

Should be simple, given that I'm overthinking it. Thanks.

That's certainly not the case for making STRICT tables the default.

But figuring that out requires understanding how they are implemented, or reading the forum, which the author admittedly didn't do.

The rest are assumptions about best practices (also not shared by the developers of SQLite).

OK, but this:

This should be a nice middle ground which avoids breaking backwards compatibility, but lets the database engine move forwards and not be bogged down by its own history.

Does nothing to address my criticism. Handwaving "this is a nice middle ground" does not address the issues. How does the feature work when enabled on an existing database, and on a new database?

Please understand that when you write `CREATE TABLE …` this statement is copied pretty much verbatim to the schema table. And that's the metadata that's saved for the table: a verbatim copy of the DDL.

If you don't mark it STRICT any current version of SQLite will consider it not to be.

So if you make assuming tables are STRICT the default (even with a PRAGMA), you'll have to deal with tables that you assume are STRICT, but aren't: they have invalid types, or invalid data.

The STRICT feature, as it was added, is backwards compatible in the sense that: (1) all old databases work with new versions of the library, and (2) all new databases fail fast (before corrupting data) in old versions of it, as they'll refuse to parse STRICT tables.

To fail fast for existing database files, you'd need to integrity check the entire databases.

So how do you implement it, what's the alternative, exactly? Add STRICT to every table you create since enabling the PRAGMA? You could do that but, to my knowledge, it'd be the first time you'd do that (modifying the schema before writing it) in 25 years of SQLite.

Whatever you come up with must be backwards compatible, as that is the promise SQLite developers have made.

I get a quota of GitHub Copilot for free.

From all the models available to me I'm most happy with Kimi K2.7 (given the cost/performance).

The section "The solution: editions?" in the article addresses directly the point of existing data.

I'm sorry, where?

SQLite schema is stored as text. If you change the default interpretation of CREATE TABLE with a PRAGMA, your existing tables become STRICT, but (1) they might now have columns with invalid types (which means you have an invalid schema, and your database fails to open), (2) they may have invalid data for their strict types (which you can only figure out with a full table scan, PRAGMA integrity_check will complain).

This was discussed previously on the SQLite forum, you can read the team's position there: https://sqlite.org/forum/forumpost/0248dcf7f0ece9fb

Regarding busy_timeout, why is 5s specifically a better default? You did not engage with my argument: that 5s is no different from 1s or 60s. How do you decide?

Also discussed in the forum, with the team laying out the rational; https://sqlite.org/forum/forumpost/f0da30efa661bd9c

I think the minimum is considering the arguments by the people who promise to maintain the software for the next 25 years.

PS: I actually really like the idea of `CREATE TYPE alias = type` for use with STRICT tables. I would champion that feature request on the forum. Given how schema is saved, I disagree with making it the default. Having to mark your tables STRICT is not such a burden, IMO.

This changes one default that "everyone agrees about" and which you can change with a compile time option: SQLITE_DEFAULT_FOREIGN_KEYS

Then it argues for STRICT tables, recognizing that there are drawbacks without introducing a new feature (custom type aliases, CREATE TYPE alias = base).

If also doesn't even considering what it means for existing data to make tables strict, which is precisely why “there is no pragma to globally make all tables strict”.

Then it argues for setting a busy timeout, and picks 5s. Why? Why 5s and not 1 or 60s? SQLite doesn't decide, which makes perfect sense. Your OS or programming language also doesn't offer you locks with a default timeout: it's either indefinite, or an instant "try lock".

Finally: WAL mode is a different file format, unsupported on many platforms, in more danger of silent corruption. Why should it be the default?

Is a dumb sea mine that blows up everything somehow better than a smart sea mine that knows to not blow up sometimes?

Yes. If smart mines lead to belligerents laying more of them because they're "safer."

Stuff that cares about security fixes this, though:

https://go.dev/blog/path-security

The functions Command and LookPath look for a program in the directories listed in the current path, following the conventions of the host operating system. Operating systems have for decades included the current directory in this search, sometimes implicitly and sometimes configured explicitly that way by default. Modern practice is that including the current directory is usually unexpected and often leads to security problems.

https://pkg.go.dev/os/exec#hdr-Executables_in_the_current_di...

Precisely, this is my experience as well.

If you use strict tables with my Go SQLite driver, you'll get worse support for bool/date/time columns than otherwise.

It's still unfortunate though that typing a column DECIMAL triggers numeric affinity, which destroys decimal numbers stored as strings.

A comment is not something I can get programmatically through `sqlite3_column_decltype` or `sqlite3_table_column_metadata`.

I can use either API to trigger bool and time handling in my Go driver.

https://github.com/ncruces/go-sqlite3/blob/main/driver/drive...

I always loved the "syntax" of the "beautiful report language" and how the [] meshed with Scheme's () and HTML/XML <> (and maybe JSON {}).

https://brl.sourceforge.net/

The downside is that you loose the place to store the metadata that a column is supposed to store a date.

Which is why I prefer not to use them.

Only if you're not compiling yourself, otherwise, there's SQLITE_DEFAULT_FOREIGN_KEYS=1

They are not, especially after twilight, or before dawn.

Also can the code even be copyrighted?

For my machine translation of SQLite to Go I added this to the README as to licencing:

Most of the code here is machine translated using wasm2go. As such, the original authors retain copyright and the original licenses remain in effect. Everything else is licensed under MIT-0.

The translator (wasm2go) has a licence chosen by, and a copyright notice from, me. Makes no sense for the translated code.

Has the dictator been removed for it? Someone cherished by them? Or are we to assume those are impollute?

It could be worse can only take a government so far. Eventually, just preaching to the choir catches up with you.

The first word in your comment spoils the rest of it.

That's … exactly what they're doing. This is the outcome.

And all it takes is not blindingly accepting the first thing it spews if you suspect there's a better answer (and are in a position to evaluate that better answer).

I'm using this new programming language: it's called LLM prompting, and everything is undefined behavior.

And the fixed version of that screen (a few tweets later), simply removes the copied text.

Which goes to show there was never any original work to fallback to.

"See, if I remove the detailed descriptions - and the LLM regurgitated the rest - nothing will have been copied."

Some people really want to defend "build me a copy of thing because I don't like the license" to be acceptable behavior.

If you build flattened a vector of them (as they argue), it can approach a byte code interpreter, though it won't be a very dense vector, if it holds "pointers" (that you need to chase) to the instructions instead of the instructions themselves.

A lot of the slowness of interpreters (and why JITs work) comes from the fact that you're executing (and trying to predict) the interpreter's branches - not the branches in the interpreted code.

This doesn't move the needle there, at all.

Should population density in cities also be measured in hab/km³ rather than hab/km²?

We are used to stacking people vertically in cities.

You might be surprised to learn that the work of a copywriter is also copyrighted.

Using acronyms is one of those ways. /s

I'm using this other cool visualization [1] for my sorting repo [2] (thanks invzhi!)

1: https://github.com/invzhi/sorting-visualization/

2: https://github.com/ncruces/sort

Those choices (what counts as a step) are interesting because e.g. for my quicksort they make it look median-of-3 is about as fast as median-of-ninthers (it's not it's significantly faster), which would make my strategy of starting with median-of-3 and doing a round of ninthers for pathological input a bit absurd.

The visualization was still incredibly useful (I found) in getting a sense that the algorithm is working correctly. You can clearly see how median-of-ninthers is working in that gif.

This is a good point.

If we don't care about inspecting the output, why don't we just replace the Rust compiler with something that generates assembly from Rust source, and then cross check the output with a few million random Rust programs?

You have a grammar file in a formal language, and want to generate a faster parser in another formal language.

What's wrong with the source language that it's better to use a sufficiently smart random code generator for the target language, and then fuzz the hell out of the output of it until it behaves the same as the slow translated code, than to create a sufficiently smart compiler from the source to target languages?

I mean this sounds like if we replaced GCC with a really smart random assembly generator and a fuzzer for the output.