armcat

This looks great, will try it out. I had a crazy idea of doing a full agentic demo video narration and cutting, so might try your tool with some tweaks.

TTS has come incredible long way, there are so many options. There is Kokoro of course, then there is Pocket TTS which is also a tiny 100M model that allows voice cloning. There is also Chatterbox Turbo, which is bit bigger but also allows for more emotional control of the voice. And then finally there is the Fish Audio S2, which is even bigger but allows even larger and essentially unbounded finegrained control of tone and emotion. And all of these can easily run on your Macbook.

Does anyone know if this happens in the Claude desktop app?

I find it astounding that ppl still comment “it’s still behind” or “it’s not the best model”. Everything is about the harness. Even the big AI labs are focusing on managing agents - sandboxes, memory, context, skills, loops. With the right harness GLM 5.2 can do no wrong.

I have the opposite experience with this, and I personally also default to simplest solutions at least as a baseline. However it’s important to distinguish between simple solutions that approximate the problem very well, to simple solutions that work in limited context or with heavy sacrifice in assumptions because those will hurt you in the long run.

I mean it's always nice to play around with sLLM finetuning, but for practical purposes I would always start with a lazy learner using embeddings (something like a small Stella model), pre-embed the topics/categories, embed the question, perform a kNN using cosine distance. You can use an LLM to "expand" the topics before embedding to make them more contextual. This is usually super fast and super simple and gives you a nice baseline. Then I would add a classification head after embedding layer (with maybe some dropout + 2-3 MLP layers) and train my own classifier, and compare that to lazy learner. Only after that would I start finetuning an LLM.

That movie has aged incredibly well!

Neko Health has been doing this now for a few years. What I heard is that ultimately it doesn’t solve much (other than them privately collecting all your data) because there are lot of false positives and these false positives are deferred to the general healthcare system, which is a major bottleneck.

Can this be used for ML/AI projects as well? I'm thinking for version controlling LoRa finetunes, finetuning data (which can consist of text, images and audio), safetensors, etc?

As an amateur watercolour artist (shameless plug: https://www.instagram.com/p/DBlKG5cMPxa) I have to say the feeling your made with this wash is gorgeous. Back in the analogue world - paper grain and type/brand has a lot to do with it. Watercolour is really about unpredictability - it's about taking advantage of this unpredictability in terms of how the water travels down the grain and the impact that it makes, combined with light/shadow and "confidence" the artist brings with the brush. So of course it's never going to be truly transferrable digitally, but I still love the work you put into this.

I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 (https://www.ai.se/en/project/gpt-sw3) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by its people, basically for free. Nations should mirror what Cursor has done with Composer 2.5 for example.

This looks beautiful and I'm sorry the current state of affairs has made you not want to publish the code, I would love to play around with it. Regarding your decision to build - I feel you, I've had the same happen to me for everything from charting libs to various web components.

As an aside, I really like your web page - simple and clean with images and demos, no bloat.

How quickly they get new models supported on the API and it just works, is insane!

This person did a great comparison against Qwen models, and despite them having 8x less active params, they outperform the Cohere model in every category: https://x.com/DJLougen/status/2057196012918149368?s=20

What an awesome story. Not too many stories about Aussies out there, but what Han brothers are doing with Unsloth in AI, and stories like this one, makes this fellow Aussie super proud!

Interesting concept! A suggestion: `whichllm <USE_CASE>` would be more beneficial, i.e. `which coding` or `which text-to-video`.

Sun Solaris PPC (CDE) takes me back. I've built plenty of 3G/WCDMA telephony code on that thing. It never let me down.

Any particular reason for BM25? Why not just a table of contents or index structure (json, md, whatever) that is updated automatically and fed in context at query time? I know bag of words is great for speed but even at 1000s of documents, the index can be quite cheap and will maximise precision

Is it OpenAI Cowork?

As someone who's been working in legaltech space where MS Word add-in chatbot was a killer feature, this is brutal. And in their demo they are hammering on the legal case (redline chat).

I still find it incredible at the power that was unleashed by surrounding an LLM with a simple state machine, and giving it access to bash

Not on the same extreme level, but I know that some coffee machines use a tiny CNN based model locally/embedded. There is a small super cheap camera integrated in the coffee machine, and the model does three things: (1) classifies the container type in order to select type of coffee, (2) image segmentation - to determine where the cup/hole is placed, (3) regression - to determine the volume and regulate how much coffee to pour.

This is beautifully written and visualised, well done! The KL divergence comparisons between original and different quantisation levels is on-point. I'm not sure people realize how powerful quantisation methods are and what they've done for democratising local AI. And there are some great players out there like Unsloth and Pruna.

That's sick, looking forward to it! You have my email in the profile, please let me know when you do!

This is awesome, well done. Been doing lot of work with voice assistants, if you can replicate voice cloning Qwen3-TTS into this small factor, you will be absolute legends!

Great work! Kind of reminds me of ell (https://github.com/MadcowD/ell), which had this concept of treating prompts as small individual programs and you can pipe them together. Not sure if that particular tool is being maintained anymore, but your Axe tool caters to that audience of small short-lived composable AI agents.

Sure, feel free to reach out, just check my profile!

They should! If you take Parakeet (ASR), add Qwen 3.5 0.8B (LLM) and Kokoro 82M (TTS), that's about 1.2G + 1.6G + 164M so ~3.5GB (with overhead) on FP16. If you use INT8 or 4-bit versions then are getting down to 1.5-2GB RAM.

And you can always for example swap out the LLM for GPT-5 or Claude.

I really like this, and have actually tried (unsuccessfully) to get PersonaPlex to run on my blackwell device - I will try this on Mac now as well.

There are a few caveats here, for those of you venturing in this, since I've spent considerable time looking at these voice agents. First is that a VAD->ASR->LLM->TTS pipeline can still feel real-time with sub-second RTT. For example, see my project https://github.com/acatovic/ova and also a few others here on HN (e.g. https://www.ntik.me/posts/voice-agent and https://github.com/Frikallo/parakeet.cpp).

Another aspect, after talking to peeps on PersonaPlex, is that this full duplex architecture is still a bit off in terms of giving you good accuracy/performance, and it's quite diffiult to train. On the other hand ASR->LLM->TTS gives you a composable pipeline where you can swap parts out and have a mixture of tiny and large LLMs, as well as local and API based endpoints.

This is an outstanding write up, thank you! Regarding LLM latency, OpenAI introduced web sockets in their Responses client recently so it should be a bit faster. An alternative is to have a super small LLM running locally on your device. I built my own pipeline fully local and it was sub second RTT, with no streaming nor optimisations https://github.com/acatovic/ova