Aspen – Local LLM for Mortals

https://www.runonaspen.com/

by mayankm • 1 month ago

2 3 1 month ago

mayankm2• 1 month ago

Hi all! We've been burning endless tokens coding on Claude and Codex, especially running agents 24X7 for our Dark Factory (per Justin @ StrongDM).

I've also personally been a bit freaked out about all the data going into Claude - everything from personal to professional to kids to finances to healthcare and more. I don't fully trust that companies that haven't had time to mature and yet have IPO-level revenue pressure coming up will continue to operate ethically and do right by my data with changing priorities, leadership, culture, and more.

Finally I also believe that everything that is currently internet-connected will be AI-enabled - this means your home robovac, coffee maker, kids toothbrush, TV, to saas, and more. And everything running through Claude is a non-start. So I built Aspen to make it dead easy for us mortals to use open-source LLMs.

Aspen makes it so that: - It always uses the best open LLM based on your hardware - You have a companion iPhone app with chatgpt.com like functionality - No data leaves your phone + machine - You don't pay at all - no tokens, AI 24*7 - A companion web-app that allows anywhere / anytime access with plenty of Claude-like functionality built into the app (artifacts, coding, design, etc) - Always updates to the latest / greatest automatically - You can issue API keys and replace your Claude / OpenAI keys for different apps

Just like every home / office has a phone, TV, and laptop, I truly believe every home and office will run their own "Aspen" for 90% of what they need and 10% go to a frontier model as needed.

Would love any and all feedback to mayank DOT mehta AT gmail DOT commerce. Thanks alL!

ai_critic• 1 month ago

Do you have any sort of hardware specs? Architecture? OS?

mayankm2• 1 month ago

Mac: Apple Silicon (ARM64), M1–M4. Windows: x64 (Intel/AMD)

4-8B param models work on laptops but the output isn't great. Gemma 12B is decent and 36B is pretty good but both need a Mac Studio or above.

Aspen – Local LLM for Mortals

Related Stories