cataflutter

This is going to be very geographically dependent, surely?

Some of the inference engines can process multiple requests in parallel more efficiently than doing them sequentially. Not sure of the exact mechanism but e.g. llama.cpp's llama-server can do this (you tell it the number of slots to have when starting, then fire HTTP requests at it and it batches them together when it can).

Waiting for the hooman (or tool calls) won't help either, of course.

They have a presence in London; have met someone who works there. Sounds like there is an office too

Disk space is one thing, but the actual download size is higher than some people's data allowances altogether! It baffles me that a lot of people don't seem to be aware of this

Disagreed; it's not a download you'd expect and it's also at least an order or two of magnitude than you'd expect to find reasonable for browsing a page.

I have a 2GB mobile data plan. If I was using Chrome, then some site triggers the Prompt API, that will cause Chrome to not only wipe out my data plan, but need 2 of my data plans. I don't find this reasonable.

This is exactly a consent problem, because I'm not denying it might be a useful feature, but it should be at the user's own informed choice. The fact that Chrome developers don't appear to see this might be due to them living in a bubble where they've never had to think about the costs.

Probably explains it, but one wonders why this page would need WebGL at all. It's not like it's a rich graphical application... is it? :)

Worth noting that NVIDIA confidential computing and similar schemes have been compromised and shouldn't be relied upon if it really matters. See https://tee.fail/ and similar.

the function signature is what they read, but the scars are what they need.

This feels like a golden quote. Don't know if you intended for it to rhyme, but well done :D

A while ago I checked this out and the homepage looked like it had fallen to the 'AI hype' trend, you know like how everything was 'AI-native XYZ for Autonomous Agents' at the time. I'm not seeing that now though.

Am I thinking of someone else or did you reverse on that?

they don't restrict you to using their opencode agent, you can use go in any other agent

Weird; I clicked through out of curiosity and didn't get any corruption of the sort in the end result.

I also asked it some technical details about how diffusion LLMs could work and it provided grammatically-correct plausible answers in a very short time (I don't know the tech to say if it's correct or not).

that said the documentation is rough, especially for their support for non-React frameworks

Do you have anywhere for people to follow the developments? :)

Do you have anywhere people can follow out of interest? :)

This looks really excellent, looking forward to giving it a try, thank you for sharing!

Sounds interesting, but if I may: the website is exceedingly sluggish, something like 1-2s interval between re-renders when trying to scroll the page (Firefox Linux). Not seeing any reason to explain this based on the page content but it's not happening with anything else on my system atm

edit: maybe `WARNING: Falling back to CPU-only rendering. Reason: webGLVersion is -1` from the console explains why, although I don't get why the page would need webGL

This looks very nifty, I will check it out and consider using it after I can understand the underlying primitives :) Thanks for sharing!

I was going to use Firecracker in an untrusted code execution project (not AI stuff); any chance you'd be happy to elaborate a bit on how you found it a pain to use? Thanks :)

`WHERE status = 'pending'` is not enough to avoid the performance problem, even if you have an index on `(status)`, because the index will still contain dead tuples until it is vacuumed.

Rough intuition: Postgres doesn't immediately delete rows, it just marks them as invalid after a certain snapshot version/transaction ID (and this mark is in the heap, not the indexes, AFAIK).

Every potential tuple that your index returns, Postgres needs to visit the heap to see if that tuple is alive. UNLESS all the tuples in that heap page are alive, in which case an optimisation called the 'visibility map' allows that check to skipped (relevant for Index-Only Scans, where Postgres can get all the results for your query from the index directly).

The only way to avoid the problem is therefore to either vacuum frequently enough that Postgres doesn't get any dead tuples returned from the index (that it must then visit the heap to check for liveness), or to bake in some condition to your queries that prevents the dead tuples from being returned from the index altogether. Vacuuming frequently is expensive and conflicts with having long-running transactions, so the latter option is generally the choice to go for when it matters.

[n.b. I feel I should note I am not a Postgres developer and wouldn't call myself an expert, just an enthusiast and dealt with a few problems here and there. So what I say should be taken with a grain or two of salt, though I believe it to be accurate.]

For disk usage, yes this doesn't address anything.

But for read performance (which is IMO what the section in the article was motivated by), it doesn't actually matter to have a bunch of entries corresponding to dead tuples in your index, provided Postgres doesn't need to actually consider the dead tuples as part of your query.

So if you have a monotonically increasing `job_id` and that's indexed, then so long as you process your jobs in increasing `job_id` order, you can use the index and guarantee you don't have to keep reconsidering the dead tuples corresponding to jobs that already completed (if that makes sense).

[This is because the index is a b-tree, which supports efficient (O(log n) page reads for n entries) seeking on (any prefix of) the columns in the index.]

Decent article, but some remarks:

1) It seems these two statements conflict with each other:

The oldest such transaction sets the cutoff—referred to as the "MVCC horizon." Until that transaction completes, every dead tuple newer than its snapshot is retained.

and

For example, imagine three analytics queries, each running for 40 seconds, staggered 20 seconds apart. No individual query would trigger a timeout for running too long. But because one is always active, the horizon never advances, and the effect on vacuum is the same as one transaction that never ends.

If the three analytics *transactions* (it's transactions that matter, not queries, although there is some subtlety around deferred transactions not acquiring a snapshot until the first query) are started at different times, they will have staggered snapshots and so once the first completes, this should allow the vacuum to advance.

2) Although the problem about this query:

    SELECT * FROM jobs
    WHERE status = 'pending'
    ORDER BY run_at
    LIMIT 1
    FOR UPDATE SKIP LOCKED;

There is a little subtlety around how you guarantee that the column is monotonically increasing, given concurrent writers, but the answer to that depends on what tricks you can fit into your application.

3) I almost want to say that the one-line summary is 'Don't combine (very) long-running transactions with (very) high transaction rates in Postgres'

(Is this a fair representation?)

I have now been waiting 2 months for a response to a similar problem. Thanks for the reminder about it, it's time to dig out the chargebacks...

Maybe what the parent commenter was referring to is that recently CloudFlare have published multiple vibe-code demo blog posts whilst trying to pass it off as production grade, for example https://news.ycombinator.com/item?id=46781516 where they implemented an open standard communication protocol on CloudFlare Workers.

The blog post was chock full of factual errors, claimed to be based off project X but wasn't at all and even had the cheek to include that it was arguably the most secure way to deploy such a server, with their implementation apparently already being used by their team to serve real traffic. Meanwhile the repo was full of TODOs for all the security aspects of the protocol.

Of course after the backlash a lot of this was covered up so look at the archives if you are curious.

They have really done a disservice to themselves because their blog posts used to be excellent, but now I have to question whether it's another blogpost full of fakery like that one (and there was another since iirc). Given this blog post talks about reimplementing a popular project, it starts to give off the signs of being another one of these. Unfortunate if that's not the case