ronfriedhaber

is broken on mobile

Not true as of last deployment.

but zero technical detail

What additional technical detail would you be interested in?

Also seems a couple orders of magnitude slower than what I’d expect even for a single core.

1) it's more than 2m/s on a MacBook AIR 2) Most of the latency is networking. This was a preliminary setup with a client / server over websockets where the book of 1m contracts combination of (strike, exp_date). Each insert only after a corresponding query has been returned (over the wire) from the previous insert. I can agree the eval isn't great; as in, the throughput is much higher.

What exact transformations are being done

This is provided in the blog post.

One man's baller is another man's insufficiently baller.

Email me for details, pricing & installations, or your target use case, would love to talk. In addition, if you have any feedback.

ron at modolap dot com

This is cool.

As far as I can tell, incremental the library aims to solve the problem of partially hydrating a computation graph when source data is altered. This approach is similar to the one pursued by (well designed) build systems and is common in the FP world. [2] This has many use cases and is very cool.

In addition, in the sphere of incremental computation, there exists Differential Dataflow, Timely Dataflow (adjacent), and DBSP. Systems like Feldera are built on DBSP. Materialize is lead by some DD guys.

Personally, I am pursuing an orthogonal approach specifically for the problem of financial data and financial workloads, There exists huge, very important problems to solve! [1]

[1] https://modolap.com

[2] Signals And Threads episode on the subject https://signalsandthreads.com/build-systems/

I tend to agree. This is just the intersection of two age-old problems; Kelly Criterion (pareto-optimal sizing of bets) and Organizational Management.

Perhaps what separates the good companies from the Great companies is that the latter always tries to make it works, doesn't rest on it's laurels, and takes any bet which is +ev.

The great Jarred Sumner pulled it off with bun, whether it can be pulled of with Postgres is an open question..

DST systems such as Antithesis can definitely help.

this is all so funny, gl to everyone

CDC which is error prone

Just have superior CDC :)

Appreciate the feedback. I shall certainly revamp the README; it is rather stale.

"how Modolap differs from just asking AI to use any other OLAP engine"

There presently exist two components, the OLAP query engine and the remote infrastructure service. The service enables systems like Codex (or developers as well) to manage datasets, maintain version control over queries, and offload the computational burden to dedicated machines. This is especially beneficial given the current trend of running agents inside micro-VMs.

In addition, it is designed with AI usage in mind. There is significant value in co-design. One could argue that models can use Polars or DuckDB just as well, and that there is no room for improvement, but I do not think this is true.

Pandas is terrific, yet even its original author has noted inherent shortcomings [1], and there exist alternatives.

Polars seems to be the most prominent competitor in the Python DataFrame space, and DuckDB appears to pursue an approach similar to SQLite, but columnar.

I am personally working on a solution to a broader problem, which can also be viewed as an alternative to Pandas [2].

[1] https://wesmckinney.com/blog/apache-arrow-pandas-internals/

[2] https://github.com/ronfriedhaber/autark