LLM App Stack – a.k.a. Emerging Architectures for LLM Applications

https://github.com/a16z-infra/llm-app-stack

by sebg • 2 years ago

54 10 2 years ago

menacingly4• 2 years ago

I sure hope the hype cycle doesn't destroy what's truly valuable here.

I accidentally stepped into seemingly endless demand for AI projects this year, and it has been a god-send for my crushing burnout. Truly interesting stuff that makes me feel like I felt when I was a kid learning unix.

I can accomplish such objectively cool things that add measurable value with these new tools, but the hype makes it all feel mildly gross.

I am both excited and feel the need to wash my hands.

sshaginyan• 2 years ago

Could you describe the types of projects you worked on?

menacingly4• 2 years ago

I have a close friend who is constantly hustling but has little technical expertise, so he just says "yes" and hopes I'll swoop in and figure out execution.

We built a lead qualification and sales coaching plug-in for a CRM, the thing has a lot of personality. IMO it was a project conceived to automate sales that ended up automating sales coaching.

From there I've done a fair bit of red teaming customer-facing chatbots, then helped save some stuck projects because these approaches to problem solving are so new.

Something that was really satisfying was a project to introduce natural language queries to a ridiculous array of internal corporate databases without sending private data out of the network. That thing is like an extremely boring corporate Jarvis.

I have absolutely zero traditional "AI" skill set, but I guess there's a lot of room downstream for implementation engineering.

moltar• 2 years ago

endless demand for AI projects

If you need any help, let me know!

I'm a freelancer and have been doing various LLM projects too, using RAGs (Zep, Bedrock KB), promptfoo (prompt testing).

A lot of it is deployed via AWS CDK, using serverless components, such as Step Functions - great for orchestration.

Using TypeScript end to end.

menacingly4• 2 years ago

grabbed your email, I'll try and get in touch next week

hnyoyo• 2 years ago

More like a16z talking their own book. It’s too early to be defining a stack so concretely.

sa-code• 2 years ago

This doesn't mention text-generation-inference, vllm, or Mistral which is raises eyebrows. I would like to see more importance given to privately deployed LLMs that are not consumed through third party APIs

LLM App Stack – a.k.a. Emerging Architectures for LLM Applications

Related Stories