vkkhare

updated the link

Isn't the inference cost of running these models at scale challenging? Currently it feels like small LLMs (1B-4B) are able to perform well for simpler agentic workfows. There are definitely some constraints but surely much easier than to pay for big clusters on cloud running for these tasks. I believe it distributes the cost more uniformly

We will be open sourcing the entire platform soon. This blog shows our platform to build the AI app

https://www.nimbleedge.com/blog/how-to-run-kokoro-tts-model-...

We have just started open sourcing the on-device AI platform.

We have started with the github repo for custom Kokoro TTS model. It is basically a batch implementation for Kokoro while supporting streaming.

https://github.com/nimbleEdge/kokoro

We will soon share the discord community too.

maybe try local p2p networking and compute offloading?

Check out https://nimbleedge.ai for a cool demo

This is our repository https://github.com/NimbleEdge/RecoEdge

I would love to talk more about it and understand your take. Is there a way I can reach out to you? I have been reading the content in web 3 and form my own opinion on the merits/demerits.

What do people think of automatic differentiation support facebook was trying for Kotlin.

They called it differentiable programming https://ai.facebook.com/blog/paving-the-way-for-software-20-...

I was thinking along ML. How analytics and ML happens on user data which is stored on data silos. For example, twitter has data about all our posts and interactions and uses that data to build recommendation models which it monetizes as to who comes on top.

In decentralized world, everyone can build their models, host them and train them on user data. But since, I wouldn't want my data to be used for any other purpose, I don't want them to make a copy of that data. So they send their models (after taking my consent) to my device, train it locally and send the aggregated model back.

I believe edge computing is one of the game changing technologies for the decade. I am not sure if it can fall under the purview of Web 3 or not. For example one of the library I implemented was along training ML models on user devices instead of the cloud maintaining privacy and personalization.

https://github.com/NimbleEdge/RecoEdge

you can find many such people in our discord server https://nimbleedge.ai/discord

We all care about how our data is used and processed.

That's why not giving them ability to copy data, allow them to use that data only on your device and send aggregated information back. So they cannot copy the data

That's what I am worried about, from its description the intent looks good but being aligned to only one industry or vertical defeats the purpose of ubiquity. People wouldn't trust it if they all just see is a volatile currency

I do care if that data gets leaked or if that data is used by third party companies for things I have bought and I do not want them to know. Why should a bank do my credit scoring based on my what am watching on Amazon or what books am I reading

Marc is reading this ;p

This is a very cool description! Hits to the point

Exactly if that is the case, it should also include technologies where data ownership lies with the users. After all data itself is a huge resource. One of the biggest reasons corporates could monopolise the platform because as users we only had our own data but no access to others data.

Maybe a consent mechanism to use other's data in privacy safe manner, where anyone can build their analytics models and people can choose which one to follow or choose. Seems like a interesting proposition.

Web 2 as a term itself was coined much later when people needed to distinguish web 3 from status quo

Edge computing for machine learning. Instead of running ML models on the cloud, I train them on user's device, ask these devices to offload computation between each other and give me the best performance out there. One good example is recommendations that work offline for you. Imagine you are listening to spotify in offline mode (with you downloaded playlist) and recommendations adapt accordingly even without internet!

I built out the library for these myself, checkout https://github.com/NimbleEdge/RecoEdge

Makes sense though I wonder what would be the original source of that data (someone like google/microsoft must be logging user data and then making some parts of it anonymized and public).

Maybe also look into on-device learning, it can be efficiently hooked up with differential privacy and give more specific results.

How do you train newer models then? From what I read you use public datasets to train your models but what about in future? You would need some kind of data collection mechanism?

Gpt-2 and gpt-3 are great but the datasets they are trained would soon get old.