Ask HK: How are you building AI apps today?

https://news.ycombinator.com/item?id=48323554

by Mnexium • 2 months ago

7 10 2 months ago

news.ycombinator.com

Generally curious how folks are using LLM APIs today.

Are folks doing anything bespoke, unique or different?

I think we've seen a lot of usage with harnesses, CLIs, Cursor/Windsurf etc which uses the APIs underneath.

If you are using the API itself, what does your architecture look like with said API?

late_night_fix• 2 months ago

I try to batch requests and reuse responses wherever possible.LLM API calls are awesome,but if you don't watch your usage,cost sneaks up fast.

Mnexium3• 2 months ago

Agreed, are you doing anything to take advantage of cacheing?

Anything else you can share about project?

LogicCraft678• 2 months ago

Mostly multi-model routing and structured outputs

sibidharan2• 2 months ago

I am using Agents SDK by OpenAI. It's interoperable with every other inference provider, even local models running on LMStudio.

I am using OpenAI from the beginning for all AI Apps I build. Initially it was only Completions API. Then came Responses API. They introduced something called Assistants API with conversation stored on server side and soon pulled the plug on Assistants API for the enhanced Agents SDK with all sessions and things stored locally as we want.

So I moved all my old completions/responses API projects to Agents SDK! They feel good and stable. Making chat with Agents SDK is super easy. Can stream tool calls and tokens effortlessly!

Agents SDK takes care of sessions, token tracking, caching, and so many things! In my apps, it helps me track how much is cached, how much is new!!! And best part about Agents SDK is it takes care of cleaning up old tool calls that saves your context, and it also auto summarises as the chat grows (I might be wrong about the last one).

I am building an EdTech with lot of AI learning / evaluation tools including isolated compute layer for my students! That led me to create an OSS project - which might have some answers for your original question.

I am working on an Open Source Async SAPI for PHP to make PHP convenient for building realtime AI apps (still in alpha and actively developing), and have created a small lesson on how to use agents SDK for AI Apps as a way to showcase my framework. If you like to see my approach, this lesson is a good place to skim.

Lesson 29: https://php.zeal.ninja/learn/ai-chat

Agent SDK Example Code used on above lesson: https://github.com/sibidharan/zealphp/blob/master/examples/a...

I follow this style everywhere in my code. Agents work as separate python code detached from whatever framework we use to build Apps, streams via STDIO and I stream the tokens over SSE/WebSockets to frontend as needed - clean architecture.

Different architecture may have different needs! A simple chat response, SSE is ok. A complicated long running stream, WebSockets!

This is how I am doing. Interested to know how others are approaching this.

Mnexium3• 2 months ago

Good write up - I think a lot of peoples journeys (at least my own) is completions api --> responses --> deeper into the tech.

How useful have you found the Agents SDK? Meaning the SDK taking care of sessions, chat history etc. is that very useful or just "its nice that it is there..." could have built it on your own if need be type of feature?

Much Luck on the EdTech tool - interesting on the PHP choice. Just personal preference?

sibidharan2• 2 months ago

Agents SDK is very useful. In-fact can design handoffs and multi agent orchestration very intuitively, and that is where the sessions, chat history all comes together. Agents SDK solves all the caveats with Responses API, from making it more than an API, into an SDK. The SDK is a layer on top of Responses API, so the SDK gives you lot of boilerplate for so many things.

I love PHP since beginning! ZealPHP is my own creation, actively developed. We created it inside our Academy. It powers our EdTech tool today as well. We initially used Apache+mod+php, but as AI needs grew, I don't want to rewrite the whole codebase for asynchronous operations to have SSE, sockets, streaming etc. I used to do with sidecars like RabbitMQ. So it began as an experiment inside the academy, today the same framework powers the EdTech tool we use in-house. To be exact, we made the same Apache+mod_php codebase run in ZealPHP and made it asynchronous.

And why PHP... because it's bloody fast when run asynchronous! HTML rendering is first class, we made ZealPHP yield HTML like react's renderToPipeableStream, and much more using OpenSwoole, making it close to the SAPI contract with classic PHP mental model. So I wanted to explore what PHP could offer in this new direction.

If you like to explore more on how PHP is "bloody fast" and "why": Kindly check https://php.zeal.ninja/performance & https://php.zeal.ninja/why-zealphp

If you like to explore how we made the same codebase run on 2 different server: https://php.zeal.ninja/case-studies/sna-labs

// ZealPHP being Actively developed now!

ApiFB-Dev• 2 months ago

Currently using multiple different ais, restrictings high rate usage and creating for scalability!

Kuyawa2• 2 months ago

I use a custom coding agent for terminal that uses file tools to read and write to my local file system and nothing else, it connects to DeepSeek API.

I start with a prompt to build an app and I enjoy the show as it spits line by line its thought process and in a couple of minutes the final result.

Mnexium3• 2 months ago

Curious to learn more. How well does file system work? At what scale are you operating? Do you see it scaling well into the future for your needs?

It seems local file system is the default for most folks starting out. I am curious to know when does this break down if ever.

Kuyawa2• 2 months ago

As a solo developer my coding agent is all I need to develop my own apps. It does 90% of the job in 5 mins for apps that would have taken me a couple of months in tedious tasks like routing, crud, auth, themes, admin dashboards, etc.

Take a look for yourself

https://www.npmjs.com/package/mecha-ai

https://github.com/kuyawa/mecha-ai

Ask HK: How are you building AI apps today?

Related Stories