Ask HN: Is anybody selling API access to HuggingFace models?

https://news.ycombinator.com/item?id=36644255

by JuanPosadas • 3 years ago

3 7 3 years ago

news.ycombinator.com

I've used the OpenAI and GCP chat-bison APIs with great success, but I'd like to put my money into more ethical companies with free-as-in-freedom-er models.

I don't want to fickle around with buying or renting GPUs or installing random LLM scripts that don't work 90% of the time.

I also don't want to pay $XX/month subscriptions, I want clean API billing like OpenAI and GCP.

What exists? Is anybody doing anything?

smoldesu2• 3 years ago

Many of the "free as in freedom" models are not trained or licensed for commercial use. For example, selling access to a LLaMA API would be a direct violation of the license the model was provided with.

version_five3• 3 years ago

I don't think you mean free as in freedom. Maybe freemium, I can't think of a closer short description. They don't cost anything but you're not allowed to use them for most stuff either.

Otoh, there are plenty of Apache and MIT licensed models. For example Open LLaMA and Falcon, Dolly, etc. HF lets you filter by license.

smoldesu2• 3 years ago

You're right. In fairness, the actual code for training LLaMA is "FOSS" technically. It's just that the model license (and weights afaik) are noncommercially licensed.

I guess it would be cool to see a paid Falcon/Dolly API, but in my experience those models get smoked by a good LLaMA 7b tuning. You would really have to hate OpenAI to seek out a non-GPT3 solution here...

version_five3• 3 years ago

Sort of related, I'm surprised there are no "edge" GPUs available in the style of cloudflare workers or AWS lambda. I heard that cloudflare was going to do something like this but it never materialized. It would be great (as an alternative to API access) to be able to pay for a worker GPU that you only used when doing inference. I know there are technical challenges to that, I wonder of it's still too hard to make something responsive enough to be useful?

ilaksh• 3 years ago

What about replicate.com and RunPod?

version_five3• 3 years ago

Thanks! I was looking for something like this last year and couldn't find anything I'm glad to see progress in this space!

brucethemoose2• 3 years ago

This is not really an answer, but I really like the "AI horde" API setup.

https://aihorde.net/

Namely, you have a pool of gen AI volunteer workers constantly chewing through API requests. Host requests are prioritized, but the "downtime" of your AI worker instance is spent fulfilling other requests, which earns karma that can be used to prioritize requests from other hosts should you need more throughput.

Economically, this is way more efficient than self hosting or "renting" an instance that will be idle between requests, and you aren't paying a massive tax for a proprietary API.

Ask HN: Is anybody selling API access to HuggingFace models?

Related Stories