toebee

You're absolutely right. We used Jordan's Whisper-D, and he was generous enough to offer some guidance along the way.

It's also a valid criticism that we haven’t yet audited the dataset for existing list of tags. That’s something we’ll be improving soon.

As for Dia’s architecture, we largely followed existing models to build the 1.6B version. Since we only started learning about speech AI three months ago, we chose not to innovate too aggressively early on. That said, we're planning to introduce MoE and Sliding Window Attention in our larger models, so we're excited to push the frontier in future iterations.

not part of the license!

Thank you for the kind words! We only support English at the moment.. Hope to add more languages in the future.

Thanks you!! We personally used Quickpod and Runpod the most. But you can try it now on HF Spaces without spinning up GPUs yourself!

https://huggingface.co/spaces/nari-labs/Dia-1.6B

Thanks for the interest! We also enjoyed using E5-F2 :) You can try it now on HF Spaces: https://huggingface.co/spaces/nari-labs/Dia-1.6B

Thank you so much for the kind words :) We only support English at the moment, hopefully can do more languages in the future. We are planning to release a technical report on some of the details, so stay tuned for that!

We will try to make it work, but not sure if will be an easy task. For now, you can try with https://huggingface.co/spaces/nari-labs/Dia-1.6B

Thank you! You can add audio prompts of calm voices to make them a bit smoother. https://huggingface.co/spaces/nari-labs/Dia-1.6B you can try it here!

Thank you!! Works for English only unfortunately :((

Thanks for the kind words! We're just following our interests and staying upwind.

Thanks for the kind words! You can try it now on https://huggingface.co/spaces/nari-labs/Dia-1.6B Also, we'll try to update the Demo Page to something lighter when we have time. Thanks for the feedback :))

Yes! But you would need to put together a LLM system that created scripts from the book content. There is an open source project called OpenNotebookLM (https://github.com/gabrielchua/open-notebooklm) that does something similar. If you hook the Dia model to that kind of system, it will be very possible :) Thanks for the interest!

We're adding guides for Zero-shot voice cloning. You can try it using the second example on Gradio: https://huggingface.co/spaces/nari-labs/Dia-1.6B

We just clarified in the README, sorry for the confusion ;(

Note that the model was not fine-tuned on a specific voice. Hence, you will get different voices every time you run the model. You can keep speaker consistency by either adding an audio prompt (a guide coming VERY soon - try it with the second example on Gradio or HF Space for now), or fixing the seed.

Only two voices at the moment... We will need to upgrade the dataset to make that happen, and are considering that as one of the next steps.

https://huggingface.co/spaces/nari-labs/Dia-1.6B we fixed it!

You can try it now on https://huggingface.co/spaces/nari-labs/Dia-1.6B !!!

Thanks for the kind words :)))

We'll try to give a high-level overview when we publish the technical report!

Thank you for the kind words! We don't have plans for that yet, but you can always open an issue or RP on Github.

We have a ZeroGPU Space provided by HuggingFace up and running! Test it now on https://huggingface.co/spaces/nari-labs/Dia-1.6B

We're still experimenting, so do not have samples yet from the larger model. All we have is Dia-1.6B at the moment.

Thank you for the contribution! We'll be merging PRs and cleaning code up very soon :)

Sorry for the confusion. the license is plain Apache 2.0, and we changed the wording to "intended for research and educational use." The point was, users are free to use it for their use cases, just don't do shady stuff with it.

Thanks for the feedback :)

We are in the progress of fixing it! Thanks for letting us know :)

We use descript audio codec! I’m not sure if DAC works on iOS…

Thank you for the kind words! Dia wasn’t fine tuned on certain speaker, so you will get random voices every time you run it, unless you add a prompt / fix the seed.

The outputs are a bit unstable, might need to add cleaner training data and run longer training sessions. Hopefully we can do something like OAI Whisper and update with better performing checkpoints!

Thank you!! Indeed the script was inspired from a scene in the Office.

Thank you for the kind words <3

It is way past bedtime here, will be getting back to comments after a few hours of sleep! Thanks for all the kind words and feedback