dmezzetti

Very interesting project! Are there benchmarks for the "compatibility mode" or are all the numbers for the Gigatoken API?

The recent abrupt removal of the ability to see who has starred a project isn't a good move. Things like this certainly erode trust.

https://github.com/orgs/community/discussions/201209

I agree that Kokoro is a good TTS model.

If you're interested in an ONNX version and a permissively licensed TTS Tokenizer, I built a pipeline for that a while back: https://huggingface.co/NeuML/kokoro-base-onnx

100% agree on this.

I've been working on small local models for years with txtai (https://github.com/neuml/txtai). I've published close to 100 models that can run local for RAG, Agents, Vector Search and more (https://huggingface.co/NeuML/collections).

Interesting project. Happy to see someone who shares an interest in tiny vector embeddings models. I've worked on tiny (1MB - 4MB, 250K - 950K parameters) embeddings models called BERT Hash https://huggingface.co/blog/NeuML/bert-hash-embeddings

Keep up the great work!

Why does HN let OpenAI and Anthropic basically advertise but it throws down the gauntlet at a small developer like myself when we do "self promotion"?

Top 3 posts as of this moment are all about Claude.

Local models are great for a lot of things past just software development. We need to move towards solving other real world problems vs just building software. I've been focused on that with TxtAI (https://github.com/neuml/txtai) for 6 years now.

Well-Read Students Learn Better: On the Importance of Pre-training Compact Models

Related paper that's a good read: https://arxiv.org/abs/1908.08962

They have paid hosting - https://huggingface.co/enterprise and paid accounts. Also consulting services. Seems like a pretty good foundation to me.

This is really great news. I've been one of the strongest supporters of local AI dedicating thousands of hours towards building a framework to enable it. I'm looking forward to seeing what comes of it!

Why keep relying on API services? If you'd like your own local AI integration with open providers like Rocket.Chat and Mattermost, check out txtchat (https://github.com/neuml/txtchat).

Very interesting!

It would be great to see how it compares to Faiss / HNSWLib etc. I'd will consider integrating it into txtai as an ANN backend.

Have you considered using something like claude code / opencode?

It's been an issue for a while and it's even bigger now in the age of AI. Lots of people use security as a way to "have their moment" and don't really care about adding value.

But scaring people off from security reports also isn't a great idea either.

It's too bad that Anthropic is so hostile to open source. It's a big missed opportunity for them.

Nice application, great work!

AI Development is good for those who want to do it. But not a terminal career decision for those who don't.

Two words: Open Source.

This change was happening well before LLMs. People were tired of being yelled at and treated poorly.

A cautionary tale for many of these types of tech platforms, this one included.

The message here is good. I've now spent over 5 years in the OSS world (https://github.com/neuml). I started by picking a problem I was interested in and checking the work into GitHub. I've been extremely fortunate to have gained a following over the years.

Even with a following, most of the time when you publish it goes into the abyss. Every once in a while something hits but most of the time it takes a lot of patience and resolve. I've had some good visibility over the years from Reddit and Hacker News (though any post I make now on HN is marked as [dead]). It's not always fair and others can "pay" to get the visibility.

I've seen some of the other comments talking about the burden of OSS but I haven't felt that. I set my own agenda and fix what I want to fix. If someone wants to change my priorities that becomes a paid effort.

I love this feature. I've long used json_extract to create dynamic columns with txtai sql: https://neuml.github.io/txtai/embeddings/query/#dynamic-colu...

You can do the same with DuckDB and Postgres too.

I've used SQLite as the content storage engine for years with TxtAI. It works great. Also plenty of good add-ons for it such as sqlite-vec for storing vectors. It can take you pretty far and maybe it's all you need in many circumstances.

Looking forward to trying them out. Great to see they are Apache 2.0...always good to have easy-to-understand licensing.

Companies that consider an open source project a critical part of their infrastructure should sponsor or compensate those projects.

Also when someone finds a bug, the maintainers are under no obligation to fix it or fix it with any timeline or even debug what's going on. If someone wants an immediate response they should provide compensation.

A common misconception is that OSS developers do everything for free. They do what THEY want for free. If YOU want to change their priorities, companies need to compensate for that.

Are multiple LLM queries faster than vector search? Even with the example "dog OR canine" that leads to two LLM inference calls vs one. LLM inference is also more expensive than vector search.

In general RAG != Vector Search though. If a SQL query, grep, full text search or other does the job then by all means. But for relevance-based search, vector search shines.

I agree. I did this myself with TxtAI. It can store vectors, data, graphs and keyword indexes all to Postgres. https://medium.com/neuml/postgres-is-all-you-need-for-vector...

Fun project. I'm sure it will get a lot of interest here.

For those into vector storage in general, one thing that has interested me lately is the idea of storing vectors as GGUF files and bring the familiar llama.cpp style quants to it (i.e. Q4_K, MXFP4 etc). An example of this is below.

https://gist.github.com/davidmezzetti/ca31dff155d2450ea1b516...

Glad to see all the interest in the local RAG space, it's been something I've been pushing for a while.

I just put this example together today: https://gist.github.com/davidmezzetti/d2854ed82f2d0665ec7efd...

I've said it multiple times. I don't want to force those who use my projects to have to share their code unless they want to. But if someone wants to use GPL that's up to them. It's a choice.

Sounds about right for someone who is building on their own.