HN user
The goal is to make working with local models easier for developers, especially when experimenting with different models or sharing setups with others.
Would love to hear back from the community
Some of the features llmpm supports today:
• Install and run open-source models (almost all Hugging Face models)
• Package models with projects so environments are reproducible
• Serve models through an OpenAI compatible API
• Built-in chat UI for interacting with running models
• CLI workflow designed for developers
• Model benchmarking and rankings to help choose models Example:
llmpm serve llama3
This exposes an API endpoint similar to:
http://localhost:8080/v1/chat/completions
Hi folks, I’ve been working on llmpm, a CLI tool that makes open-source LLMs installable like packages. The idea came from the frustration of trying to run different models locally. Every model tends to have slightly different instructions, dependencies, or runtimes. I wanted something that worked more like npm or pip, install a model and run it immediately. With llmpm you can do things like:
llmpm install llama3
llmpm run llama3
Tried this out this morning. It’s a pretty powerful CLI tool for developers working with open-source LLMs.
Also, is there is way I can invoke the models or is there an API which your tool exposes?
Looks crazy! Thanks for building it, there is indeed a need for a npm or pip like package manager for AI Models.