spstoyanov

It's not only you, I am really curious about their data privacy.

Glad to see the price is going down but it's still too high for a "fast" model

The other day I was thinking about the future of programming and how our workflows change with AI and I started wondering what would an AI-native programming language look like.

So I got this idea - what if we can just write function / class definitions and let AI generate the code for us. One thing led to another and after a fun weekend I got something working.

It’s a simple python decorator that uses a function definition to generate the function code. It works for async functions and class methods, and it caches the generated code locally. There are some fun use cases like AI-generated fastAPI endpoints and agent tools.

I also realized that it somewhat incentivizes test-driven development because if you write the tests ahead of time you don’t only make sure that the generated function code works but it is generated and cached before the actual program runs.

I haven’t had enough time to do a proper comparison but I wonder if this test-driven approach produces better code vs traditional vibe-coding.

I would love to hear your thoughts. Cheers!

Nice! Explicit is better.

cool idea!

Thank you so much!

The main difference between langchain and LLMFlows is in the philosophy - langchain has a "chain for everything" approach where chains come with multiple LLM calls, opinionated internal logic, and built-in default prompts. On the other hand llmflows has a "simple, explicit, transparent" approach where the goal is to allow developers to easily build their own chains (or flows) and have full control and transparency over the apps.

If you are curious check out the intro section of this blog post I made some time ago: https://llmflows.substack.com/p/introduction-to-llmflows

This is awesome! Thank you for sharing!

brain activity

It's definitely not only you

Thank you for your feedback! And thank you for sharing guidance!

You might find this guide useful to your use case:

https://llmflows.readthedocs.io/en/latest/user_guide/Async%2...

It shows how you can connect multiple similar flowsteps to a previous flowstep and they will all run in parallel.

Thank you for the kind words!

This is a very hard problem. If you base your response on a text that has been returned from a vector DB you can reference this text. But if you rely on a citation generated directly from the LLM is probably impossible to reliably guarantee the source is real. I guess if it’s a link you can check if it’s actually a real link, scrape the content and do some extra trickery to figure out if it contains the text produced by the LLM but I’ve never tried this myself.

Makes sense! I will look into it

Thank you for your feedback. Azure OpenAI should be quite easy indeed. I’ve never tried Bard as well and have no clue how popular it is. Also I’ve never heard of Amazon Titan. Thank you for sharing!

Yes, I would love to get some feedback on what people want to see - right now I am thinking Bard and Claude. What do you want to have?

Thank you for your feedback!

Building apps with LLMs is fundamentally an exercise of manipulating strings and making API calls (Both LLM and vector db APIs). When apps are more complex and start including multiple LLM calls and prompts that can change dynamically some additional challenges start to emerge. e.g figuring out:

- When was a particular LLM called?

- How much time did it take?

- What were the input variables?

- What did the prompt look like?

- What was the exact configuration of each LLM?

- How many times did we retry the request?

- What was the raw data the API returned?

- What was exactly returned from the vector store and fed into an LLM?

- How many tokens were used?

- What was the final result for each call?

- How do you make API calls in parallel?

I think a framework like this should provide abstractions that allow people to focus on the important part like prompt engineering and productionizing the app and worry less about how to figure out stuff like this.

Right now LLMFlows supports only OpenAI models and Pinecone but I am working on classes for Chroma and Weaviate. And I would like to also provide support for Bard and Claude once I get access.