HN user

adsharma

508 karma

ladybugdb.com

Posts4
Comments366
View on HN

Many recent graph databases are built like this. If you compare ladybugdb vs duckdb, the internals are very similar.

The main innovation is the "REL table". You can think of it as a many-to-many relationship table on steroids with optimizations at the storage layer and join algorithms.

Definition of what makes something a graph database (apart from the query language) is contested. But we seem to be moving towards: build a reasonable relational database and then add a "REL table" to it with join optimizations.

Need something for backward compatible RPC? Use protobuf. But for storage and types?

This is where I feel there are better alternatives. If you go with protobuf, you're picking 2 out of 3: simplicity, fast, idiomatic.

Please consider "uvx tsc-py --help" for things that don't fit.

LadybugDB was forked from KuzuDB 8 months ago. Find out what's new and how it differs from KuzuDB in performance, features and market positioning.

A more appropriate mirror test for LLMs is to get them to state facts about their training data. Percentage of arts vs science for example.

Given the framing that they're similar to nukes and a national security issue, it's likely that the models are post trained to not answer such questions accurately.

Also the article could be trying to normalize thinking that these are more than matrix multiplication gadgets good at compression.

It's interesting that the blog post places SQLite and Ladybird on the spectrum, but omits it's chief open source rival: DuckDB.

Agree that Level 3 is what inspires confidence. But we need to invent new business models to sustain in the era of vibe-coded databases.

Rayforce 1 month ago

DuckDB and LadybugDB use the same terminology to describe internals.

The future is to write Rust in a python interpreter compatible way leveraging large parts of the python ecosystem.

This will require minor tweaks to the language and perhaps a forked interpreter (a forked parser already exists).

Then you solve the two language problem with a combination of probabilistic and deterministic translation.

+1 - I was making exactly this argument in other threads. But I have a slightly different take on how software should be written.

Translating zig -> rust is more complex than writing a JPEG parser in static python and then lowering it into zig and rust differently using idiomatic construct for each language.

Towards that end, I've created a parser for a dialect of python which is suitable for this purpose. It should maintain compatibility with the vast majority of python code out there, while picking up some rust/zig features that make translation easier. JPEG parser included in the assets of the skill for a flavor.

https://github.com/py2many/static-python-skill

https://github.com/py2many/spy-ast

PGLite Evangelism 3 months ago

The first one. It's forked from pgserver.

Yes, what you get is a multi-server postgres under the covers. But for many users, the convenience of "uv pip install...", auto clean up via context manager is the higher order bit that takes them to SQLite.

Of course the bundled extensions are the key differentiator.

With enough distribution and market opportunities we (yes, there is a for profit company behind it) can invest in making it truly embedded.

PGLite Evangelism 3 months ago

Pgserver is not maintained. Had to fork it as pgembed, compile recent versions and bundle BM25 and vector extensions.

This is why transpilers exist.

py2many can compile static python to mojo, apart from rust and golang.

Is it comprehensive? No. But it's deterministic. In the age of LLMs, with sufficient GPU you can either:

  * Get the LLM to enhance the transpiler to cover more of the language/stdlib
  * Accept the non-determinism for the hard cases
The way mojo solves it is by stuffing two languages into one. There are two ways to write a function for example.

I don't think the cost imposed by a transpiler is worse. In fact, it gets better over time. As the transpiler improves, you stop thinking about the generated code.

There is a question of what benefit would it bring even if its open sourced?

Static python can transpile to mojo. I haven't seen an argument on what concepts can only be expressed in mojo and not static python?

Borrow checker? For sure. But I'm not convinced most people need it.

Mojo therefore is a great intermediate programming language to transpile to. Same level of abstraction as golang and rust.

Amazing people still keep discovering it. And google search fails to surface working implementations.

"Python to rust transpiler" -> pyrs (py2many is a successor) "Python to go transpiler" -> pytago

Grumpy was written around a time when people thought golang would replace python. Google stopped supporting it a decade ago.

Even the 2022 project by a high school student got more SEO

https://github.com/py2many/py2many/issues/518

Thank you for the shout out! I looked into your benchmark setup a bit. Two things going on:

- Ladybug by default allocates 80% of the physical memory to the buffer pool. You can limit it. This wasn't the main reason.

- Much of the RSS is in ladybug native memory connected to the python connection object. I noticed that you keep the connection open between benchmark runs. For whatever reason, python is not able to garbage collect the memory.

We ran into similar lifetime issues with golang and nodejs bindings as well. Many race conditions where the garbage collector releases memory while another thread still has a reference to native memory. We now require that the connection be closed for the memory to be released.

  https://github.com/LadybugDB/ladybug/issues/320
  https://github.com/LadybugDB/go-ladybug/issues/7
  https://github.com/LadybugDB/ladybug-nodejs/pull/1

Makes it a good match for columnar databases which already operate on the read-only, read-mostly part of the spectrum.

Perhaps people can invent LSM like structures on top of them.

But at least establish that CSR on disk is a basic requirement before you claim that you're a legit graph database.

I didn't dismiss the language. I called it a north star. Rust is still the best option if you desire memory safety.

But rewriting a complex working piece of software in Rust is not trivial. Having an incremental path (where only parts are rewritten in Rust and compatible with C++ code) would be a good path to get there.

Also open to new code and extensions getting written in Rust.

Also an important test is the check on whether it's WCOJ on top of relational storage or is the compressed sparse row (CSR) actually persisted to disk. The PGQ implementations don't.

There are second order optimizations that LLMs logically implement that CSR implementing DBs don't. With sufficient funding, we'll be able to pursue those as well.