snakepit

Still have to update it for snakepit 0.11.0, but I did start a vLLM wrapper for Elixir

https://hex.pm/packages/vllm

This is helpful. Someone should create a similar benchmark for the BEAM. This is also a good reminder to continue working on snakepit [1] and snakebridge [2]. Plenty remains before they're suitable for prime time.

[1] https://hex.pm/packages/snakepit [2] https://hex.pm/packages/snakebridge

This doesn't reflect the current reality. Tech companies acquire questionable third-party data without consent and exploit it however they see fit.

In many ways, we're past the point of no return. So-called ubiquitous technical surveillance is largely the norm, often encroaching by design beyond the boundaries of expected decency.

Informational terrorism, a dysphemism that describes the manner by which certain data is abused to "re-rank content" for a "personalized experience," is encoded into the DNA of certain large tech companies.

AI and ML ecosystem in Elixir. Solo mission currently. Early stage. Backlogged. Open to help: contact@nsai.online

Incredibly helpful. This article helped me to identify significant fundamental shortcomings in my first attempt to design an agent/tool arbitration protocol:

https://github.com/nshkrdotcom/ALTAR

I'm working on an ML platform in Elixir. Orchestrating CPU-intensive Python processes is central to the project, so I created a session based pooler called `snakepit` [1].

The latest revision is building a batteries-included gRPC Python bridge that enables streaming, bi-directional tool use, as well as an innovative variables feature for experimental ML inspired by ideas from DSPy's team.

One of the later project goals: A Python client that can manage the Elixir orchestrator that manages pools of Python, in a distributed environment. Maybe I'll call that submodule `snakepits`. In this embodiment, it will be an effective albeit much more sophisticated replacement for `asyncio` for some use cases.

[1] https://github.com/nshkrdotcom/snakepit