A sharded DuckDB on 63 nodes runs 1T row aggregation challenge in 5 sec 9 months ago
DuckDB is not single CPU, it's single machine - big difference
HN user
DuckDB is not single CPU, it's single machine - big difference
One thing to add to this: Snapshots can be retained (though rewritten) even through compaction
As a consequence of compaction, when deleting the build up of many small add/delete files, in a format like Iceberg, you would lose the ability to time travel to those earlier states.
With DuckLake's ability to refer to parts of parquet files, we can preserve the ability to time travel, even after deleting the old parquet files