HN user

samwillis

30,110 karma

Multidisciplinary Developer, Designer and Engineer. I invent products both online and offline.

Working on sync engines, local-first, PGlite, and Tanstack DB at ElectricSQL.

sam dot willis at gmail dot com

samwillis.co.uk

Twitter: @samwillis

Posts496
Comments2,531
View on HN
electric.ax 27d ago

PGlite reaches 10M weekly downloads

samwillis
4pts1
electric-sql.com 3mo ago

AI agents as CRDT peers – building collaborative AI with Yjs

samwillis
3pts0
electric-sql.com 6mo ago

From science fiction to reality – you can build difficult things with LLMs now

samwillis
2pts0
cursor.com 6mo ago

Scaling long-running autonomous coding

samwillis
290pts197
arxiv.org 6mo ago

The Bayesian Geometry of Transformer Attention

samwillis
4pts1
medium.com 6mo ago

Attention Is Bayesian Inference

samwillis
152pts32
webkit.org 7mo ago

Concurrent JavaScript: It can work

samwillis
4pts0
frontendatscale.com 10mo ago

An Interactive Guide to TanStack DB

samwillis
4pts0
goose.icu 11mo ago

Eliminating JavaScript cold starts on AWS Lambda - Porffor can run on Lambda now

samwillis
3pts0
tanstack.com 11mo ago

Show HN: TanStack DB – Reactive DB with Differential Dataflow for TanStack Query

samwillis
4pts2
willmcgugan.github.io 12mo ago

Efficient Streaming of Markdown in the Terminal

samwillis
2pts0
trigger.dev 1y ago

We built a real-time service that handles 20k updates per second

samwillis
5pts0
mattweidner.com 1y ago

Collaborative Text Editing Without CRDTs or OT

samwillis
284pts79
electric-sql.com 1y ago

Show HN: ElectricSQL Cloud – Postgres sync in 30 seconds

samwillis
7pts0
www.theguardian.com 1y ago

Biologist whose innovation saved lives wins $3M Breakthrough prize

samwillis
6pts0
brandur.org 1y ago

Postgres UUIDv7 and per-back end monotonicity

samwillis
2pts0
babel.hathitrust.org 1y ago

Punch (1844) predicts LLMs: "The new patent novel writer"

samwillis
3pts3
badkeming.com 1y ago

Bad Keming – Where kerning's so bad it's keming

samwillis
8pts1
lucumr.pocoo.org 1y ago

Playground Wisdom: Threads Beat Async/Await

samwillis
171pts97
www.eu-inc.org 1y ago

Let's unite the startup ecosystem in Europe with a standardized ‘EU Inc'

samwillis
15pts4
jkatz05.com 1y ago

Hybrid Search with PostgreSQL and Pgvector

samwillis
35pts1
www.gov.uk 1y ago

CMA objects to Google's ad tech practices to help UK advertisers and publishers

samwillis
3pts0
www.nasa.gov 1y ago

NASA Decides to Bring Starliner Spacecraft Back to Earth Without Crew

samwillis
197pts2
open-web-advocacy.org 1y ago

UK's Browser and Cloud Investigation may fail to allow Web App competition

samwillis
2pts0
pglite.dev 1y ago

Show HN: PGlite – in-browser WASM Postgres with pgvector and live sync

samwillis
509pts108
en.wikipedia.org 1y ago

HTML Application

samwillis
1pts0
jlongster.com 1y ago

Why does the chromaticity diagram look like that?

samwillis
276pts84
evilmartians.com 2y ago

Recapping the first Local-First conference in 15 minutes

samwillis
5pts0
mattferraro.dev 2y ago

CADmium: A local-first CAD program built for the browser

samwillis
669pts241
wasmer.io 2y ago

Py2wasm: A Python to WASM Compiler

samwillis
3pts0

They have compiled to WASM64, the newer 64bit version of WASM. That's not available on iOS yet.

WASM till recently has had a 32bit address space, to match the limitations of JS numbers. With 64bit WASM you have to use BigInt rather than the floats that are numbers in js to address the memory.

It's only the address space that's affected - old WASM still handled 64bit math fine.

It's disingenuous to suggest that "Yjs will completely destroy and re-create the entire document on every single keystroke" and that this is "by design" of Yjs. This is a design limitation of the official y-Prosemirror bindings that are integrating two distinct (and complex) projects. The post is implying that this is a flaw in the core Yjs library and an issue with CRDTs as a whole. This is not the case.

It is very true that there are nuances you have to deal with when using CRDT toolkits like Yjs and Automerge - the merged state is "correct" as a structure, but may not match your scheme. You have to deal with that into your application (Prosemirror does this for you, if you want it, and can live with the invalid nodes being removed)

You can't have your cake and eat it with CRDTs, just as you can't with OT. Both come with compromises and complexities. Your job as a developer is to weigh them for the use case you are designing for.

One area in particular that I feel CRDTs may really shine is in agentic systems. The ability to fork+merge at will is incredibly important for async long running tasks. You can validate the state after an agent has worked, and then decide to merge to main or not. Long running forks are more complex to achieve with OT.

There is some good content in this post, but it's leaning a little too far towards drama creation for my tast.

Have you considered doing property tests with Mathematica as an oracle?

An ai based development workflow with a concrete oracle works very well. You still need the research and planing to solve things in a scalable way, but it solves the "are the tests correct" issue.

What we've done is pull out failing property tests as a unit tests, makes regression testing during the agentic coding loop much more efficient.

I'm convinced that LLMs results in all software needing to be open source (or at the very least source available).

In future everyone will expect to be able to customise an application, if the source is not available they will not chose your application as a base. It's that simple.

The future is highly customisable software, and that is best built on open source. How this looks from a business perspective I think we will have to find out, but it's going to be fun!

Manifold works on solid triangle meshes, OpenCascade is a true BREP kernel that represents solids as edges (straight and curved) and surfaces (not meshed) computed from those edges. There is no triangulation in the root model in OpenCascade.

This is one of the main use cases we are building "Durable Streams" for, it's an open source spec for a resumable and durable stream protocol. It's essentially an append only log with a http api.

https://github.com/durable-streams/durable-streams

https://electric-sql.com/blog/2025/12/09/announcing-durable-...

When we built ElectricSQL we needed a resumable and durable stream of messages for sync and developed a highly robust and scalable protocol for it. We have now taken that experience and are extracting the underlying transport as an open protocol. This is something the industry needs, and it's essential that it's a standard that portable between provider, libraries and SDKs.

The idea is that a stream is a url addressable entity that can be read and tailed, using very simple http protocol (long polling and a SSE-like mode). But it's fully resumable from a known offset.

We've been using the previous iteration of this as the transport part of the electric sync protocol for the last 18 months. It's very well tested, both on servers, in the browser, but importantly in combination with CDNs. It's possible to scale this to essential unlimited connections (we've tested to 1 million) by request collapsing in the CDN, and as it's so cacheable it lifts a lot of load of your origin when a client reconnect from the start.

For the LLM use case you will be able to append messages/tokens directly to a stream via a http post (we're working on specifying a websocket write path) and the client just tails it. If the user refreshes the page it will just read back from the start and continue tailing the live session. Avoids appending tokens to a database in order to provide durability.

I'm not aware of anything trying to compile timescale for it. Some extensions are easer than other, if there is limited (or ideally no) network IO and its written in C (Timescale is!) with minimal dependencies then its a little easer to get them working.

There are a few people using it in prod for customer facing web apps.

Extensions are also available - we have a list here: https://pglite.dev/extensions/. We would love to extend the availability of more, some are more complex than others though. We are getting close to getting PostGIS to work, there is an open PR that anyone is welcome to pick up and hack on.

We have a long on running research project with the intention of carting a "libpglite" with a C FFI and compiled as a dynamic library for native embedding. We're making steady progress towards it.

It's now used by a huge number of developers for running local dev environments, and emulating server products (Google firebase and Prisma both embed it in their CLI). Unit testing postgres backed apps is also made significantly easer with it.

Hey everyone, I work on PGlite. Excited to see this on HN again.

If you have any questions I'll be sure to answer them.

We recently crossed a massive usage milestone with over 3M weekly downloads (we're nearly at 4M!) - see https://www.npmjs.com/package/@electric-sql/pglite

While we originally built this for embedding into web apps, we have seen enormous growth in devtools and developer environments - both Google Firebase and Prisma have embedded PGlite into their CLIs to emulate their server products.

I'm really intrigued by the use of differential dataflow in a static site toolkit, but there isn't much in the way written about it. If anyone from the team are here I would love it if you could explain how it's being used? Does this enable fast incremental builds, only changing the parts that change in the input? If so how do you model that? Are you using multisets as a message format inside the engine?

For context, I work on TanStack DB which is a differential dataflow / DBSP inspired reactive client datastore. Super interested in your use case.

I think is really interesting how it's often suggested Waymo is at a disadvantage over Tesla due to its reliance on LIDAR and the costs associated with it. But the reality is that it's enabled Waymo to move faster and gain significant more operational experience than Tesla, and that's far more important than front loading with cost reductions in a service business.

Tesla have been operating as a product business, and cost reduction of that product was key to scale and profitability. I completely understand why they have focused on optical sensors for autopilot, lidar was always going to be impossibly expensive for a consumer product.

Waymo on the other hand have always been aiming to build a service business and that changes the model significantly, they need to get to market and gain operational experience. Doing that with more expensive equipment to move faster is exactly what was needed. They can worry about cost of building their cars later, much later.

This is a great comparison, but it depends so much on what sort of website or web app you are building. If you are building a content site, with the majority of visitors arriving without a hot cache bundle size is obviously massively important. But for a web app, with users regularly visiting, it's somewhat less important.

As ever on mobile it's latency, not bandwidth, that's the issue. You can very happily transfer a lot of data, but if that network is in your interactive hot path then you will always have a significant delay.

You should optimise to use the available bandwidth to solve the latency issues, after FCP. Preload as much data as possible such that navigations are instant.

Apple M5 chip 9 months ago

Software is very easy to bloat, expand scope, and grow to do more than really needed, or just to release apps that are then forgotten about.

Hardware is naturally limited in scope due to manufacturing costs, and doesn't "grow" in the same way. You replace features and components rather than constantly add to them.

Apple needs someone to come in and aggressively cut scope in the software, removing features and products that are not needed. Pair it down to something manageable and sustainable.

This record and replay trick is very similar to what I recently used to implement the query DSL for Tanstack DB (https://tanstack.com/db/latest/docs/guides/live-queries). We pass a RefProxy object into the where/select/join callbacks and use it to trace all the props and expressions that are performed. As others have noted you can't use js operators to perform actions, so we built a set of small functions that we could trace (eq, gt, not etc.). These callbacks are run once to trace the calls and build a IR of the query.

One thing we were surprisingly able to do is trace the js spread operation as that is a rare case of something you can intercept in JS.

Kenton, if you are reading this, could you add a series of fake operators (eq, gt, in etc) to provide the capability to trace and perform them remotely?

Given an image or a 3D mesh, SGS-1 can generate CAD B-Rep parts in STEP format. Unlike all other existing generative models, SGS-1 outputs are accurate and can be edited easily in traditional CAD software

This is a game changer, all the models before that output meshes were a toy at best. Super excited to see where they can take this.

I wander if the next step is for a step -> proprietary format (SolidWork, NX etc) model that can infer constraints.

Really interesting post, and well done on setting up the Apples to Oranges nature of the benchmark, you're very clear. It's really interesting to see the deference the district architectures make.

Did you run any tests with the new transaction system in ClickHouse? It would be super interesting to see how it effected the batch updates.

I haven't kept track of what Oliver's doing here lately

Please do go and check up what the state of using types to inform the compiler is (I'm not incorrect)

On the area allocator, I wasn't clear enough, as stated elsewhere this was in relation to having something similar to isolates - each having a memory space that's cleaned up on exit.

Python has almost identical semantics to JS, and has threads - there is nothing in the EMCAScript standard that would prevent them.