avinassh

TIL electrobun. How does it compare against electron?

I'd love to learn how the database is built and the sync works

previous Show HN post submitted by the author (109 comments) - https://news.ycombinator.com/item?id=41224689

You can shard your SQLite tables into multiple database files and query across all of them from a single connection.

You mean using ATTACH statement, right? If you use WAL mode, then you cannot get transaction safety / ACID with ATTACH [0]

If the main database is ":memory:" or if the journal_mode is WAL, then transactions continue to be atomic within each individual database file. But if the host computer crashes in the middle of a COMMIT where two or more database files are updated, some of those files might get the changes where others might not.

Moreover, ATTACH do not support more than 125 databases, so that limits the shards to 125. [1]

ATTACH does not solve the concurrency problems. That's why SQLite also has a BEGIN CONCURRENT experimental branch

[0] https://www.sqlite.org/lang_attach.html

[1] https://www.sqlite.org/limits.html

how does collaboration works for SQLite, since the db is embedded?

I have an old 2019 MBP, now I am tempted to try Pop on it. Does the external displays work?

I've been working on something similar... although with slightly larger scope (intended to be used within containers/sandboxes) https://github.com/andrewbaxter/passworth

stored in encrypted sqlite3

you had me at encrypted sqlite3. it would be great if you mention in readme that it uses SQLCipher

any opinion on how does this compare with ente?

I find this interesting. This is probably the first time I am seeing a large-ish project ban AI outright

Does substack let you add canonical URLs for the post? Otherwise won't you penalised by search engines for duplicated content?

This is cool! The code base is small to follow easily and also TIL ggez.

In the single player mode, the computer is not so intelligent. So initially i just kept winning without understanding what was happening.

Is it possible to create a filter that can work over a complex join operation?

That's what IVM systems like Noria can do. With application + cache, the application stores the final result in the cache. So, with these new IVM systems, you get that precomputed data directly from the database.

Views in Postgres are not materialized right? so every small delta would require refresh of entire view.

I don't think WAL can ever really have a durable transaction; it is essentially based on leaving a transaction open until it gets "check-pointed" or actually committed to the database file

why not? if you use synchronous=FULL, then WAL does provide durable transactions, no?

I wrote the first article, and I thought documentation is clear, but then I saw comment by Hipp which got confused me:

If you switch to WAL mode, the default behavior is that transactions are durable across application crashes (or SIGKILL or similar) but are not necessarily durable across OS crashes or power failures. Transactions are atomic across OS crashes and power failures. But if you commit a transaction in WAL mode and take a power loss shortly thereafter, the transaction might be rolled back after power is restored.

https://news.ycombinator.com/item?id=45014296

the documentation is in contradiction with this.

am now always looking for more ways to include this for inline-documentation.

same lol. here is a blog post of mine where I used them - https://avi.im/blag/2024/disaggregated-storage

I had to convert them to images because I couldn't get to working with Hugo, static site generator

hey, I just tested and `NORMAL` is default:

    $ sqlite3 test.db
    
    SQLite version 3.43.2 2023-10-10 13:08:14
    Enter ".help" for usage hints.
    sqlite> PRAGMA journal_mode=wal;
    wal
    sqlite> PRAGMA synchronous;
    1
    sqlite>

    /opt/homebrew/opt/sqlite/bin/sqlite3 test.db
    SQLite version 3.50.4 2025-07-30 19:33:53
    Enter ".help" for usage hints.
    sqlite> PRAGMA journal_mode=wal;
    wal
    sqlite> PRAGMA synchronous;
    2
    sqlite>

This looks great! I hope you will be able to get back to it

I don’t want to use JavaScript /TS but I want to make games for the web. Are there any nice framework which lets me write in Rust, it compiles to WASM for web?

is there any easier way to manage bookmarks with Obsidian? With Bases, I would get a nice UI as well

OP has this first:

If a system has distributed-consensus mechanisms, many different forms of event-driven communication, CQRS, and other clever tricks, I wonder if there’s some fundamental bad decision that’s being compensated for (or if the system is just straightforwardly over-designed).

then later down in the article:

Send as many read queries as you can to database replicas. A typical database setup will have one write node and a bunch of read-replicas. The more you can avoid reading from the write node, the better - that write node is already busy enough doing all the writes.

isn't this same as CQRS

one more difference I would add is how a leader is elected in Raft (randomized) vs VSR (deterministic succession)

huh I didn’t expect the price to be $1.99

it’s less, but good for me :)

However, SQLite, by default, always truncates the WAL files on last connection close

When the last connection to a database closes, that connection does one last checkpoint and then deletes the WAL and its associated shared-memory file, to clean up the disk.

https://www.sqlite.org/wal.html

I don't know if you have some personal vendetta against me, because you are citing things I did not say. I did not say SQLite is unreliable. I said SQLite stops at checksum errors found in WAL and stops recovery, which may lead to data loss. Which part of this is incorrect?

On SQLite contribution, I did not say it's "impossible." I said it's not open to contribution. This is the exact phrase from the linked page.

Yes. Say if I am using something like litestream, then all subsequent generations will be affected. At some point, I'd go back and find out the broken generation.

Instead of that, I'd prefer for it to fail fast

Giving developers that option would require SQLite to change the way it writes WALs, which would increase overhead.

Yes! But I am happy to accept that overhead with the corruption detection.

Yes, this is a good example of showing an we cannot partly apply the WAL always. Again, let me repeat it, we cannot partly apply the WAL all the time expect it to work but there are some valid cases where we can do that to recover. Your example is not the one.

of all places, I did not expect to get personal attacks on HN :)

yet makes fundamental mistakes in understanding what the WAL does and how it's possible not to "partly" apply a WAL.

Please provide citation on where I said that. You can't partly apply WAL always, but there are very valid cases where you can do that to recover. Recovery doesn't have to automatic. It can be done by SQLite, or some recovery tool or with manual intervention.

- Said person maligns SQLite as being impossible to contribute; whereas the actual project only mentions that they may rewrite the proposed patch to avoid copyright implications.

Please provide citation on where I said that. Someone asked me to send a patch to SQLite, I linked them to the SQLite's page.

you might like my other post - https://avi.im/blag/2024/databases-checksum/

bad news is, most databases don't do checksums by default.

As a SQL-first developer, I don't pick apart write-ahead logs trying to save a few bytes from the great hard drive in the sky, I just want the database to give me the current state of my data and never be in an invalid state.

Yes, that is a very valid choice. Hence, I want databases to give me an option, so that you can choose to ignore the checksum errors and I can choose to stop the app and try to recover.