gijzelaerr

And MonetDB took some inspiration back from DuckDB to develop an embedded version also:

https://github.com/MonetDBSolutions/MonetDBe-Python/

Disclaimer: i'm working on this.

Last year I worked with a start-up called Scentronix, which makes customized perfumes. The company made a device which can quickly blend fragrances on the spot based on 200 most commonly used ingredients in the business. Apparently, you can make most of the available perfumes with these base ingredients. The recipes are a bunch of known well-received recipes, and it will also blend new random recipes (with certain constraints like don't add too much of an overwhelming base scent). They sample feedback from people and train a collaborative filtering classifier. It is a hard problem, and the data is very noisy, but we did start to get better-than-random results at some point. https://scentronix.com/

There is doc available on the website, but I'm not sure if I explain the concept properly.

The assumption is that you have containers that operate on input files and generates output files. The behavior of the container depends on the given parameters which are defined in the kliko file. The user (or runner) will supply these parameters at runtime.

To illustrate, we use it for creating pipelines in radio astronomy where we operate on datasets of gigabyes or bigger. most of these tools are file based, they read files in and write files out. It is all quite complex and old software, so Docker is ideal for encapsulating this complexity. A scientist can easily recombine the various containers and play with the arguments. By the split of input/output the container effectively become 'functional', no side effects and the results can be cached if the parameters are the same. The intermediate temporary volumes can be memory based to speed things up. We use stdout for logging.

Interesting. I've been working on something related, a non intrusive docker 'extension', kliko. Kliko is a specification to formalize file based (no network yet) input/output flow for containers. It makes to possible to have a generic API for containers, so you can automatically create user interfaces for an application or chain them together in a pipeline.

https://github.com/gijzelaerr/kliko

Here is an example kliko file for a container defining the input and output:

https://github.com/kernsuite-docker/lwimager/blob/master/kli...

SQLAlchemy dialect maintainer here. I'm happy to hear somebody is using it, I've been working on it for a while but in the last 2 years I only received one bug report. And I don't think that is because the code is flawless.