xenova

yep! :) https://huggingface.co/spaces/webml-community/bonsai-ternary...

This demo runs Voxtral-Mini-3B, a new audio language model from Mistral, enabling state-of-the-art audio transcription directly in your browser. Everything runs locally, meaning none of your data is sent to a server (and your transcripts are stored on-device).

We have released a bunch of speech recognition demos (using whisper, moonshine, and others). For example:

- https://huggingface.co/spaces/Xenova/whisper-web

- https://huggingface.co/spaces/Xenova/whisper-webgpu

- https://huggingface.co/spaces/Xenova/realtime-whisper-webgpu

- https://huggingface.co/spaces/webml-community/moonshine-web

It took some time, but we finally got Kokoro TTS (v1.0) running in-browser w/ WebGPU acceleration! This enables real-time text-to-speech without the need for a server. Looking forward to your feedback!

NPM package: https://www.npmjs.com/package/kokoro-js GitHub: https://github.com/hexgrad/kokoro

For those interested in learning more, the source code is available on GitHub: https://github.com/huggingface/transformers.js-examples/tree...

It uses OpenAI's set of whisper models, which support multilingual transcription and translation across 100 languages. Since the models run entirely locally in your browser (thanks to Transformers.js), no data leaves your device! Huge for privacy!

Source code: https://github.com/xenova/whisper-web/tree/experimental-webg...

We have some other WebGPU demos, including:

- WebGPU embedding benchmark: https://huggingface.co/spaces/Xenova/webgpu-embedding-benchm...

- Real-time object detection: https://huggingface.co/spaces/Xenova/webgpu-video-object-det...

- Real-time background removal: https://huggingface.co/spaces/Xenova/webgpu-video-background...

- WebGPU depth estimation: https://huggingface.co/spaces/Xenova/webgpu-depth-anything

- Image background removal: https://huggingface.co/spaces/Xenova/remove-background-webgp...

You can follow the progress for full WebGPU support in the v3 development branch (https://github.com/xenova/transformers.js/pull/545).

To answer your question, while there are certain ops missing, the main limitation at the moment is for models with decoders... which are not very fast (yet) due to inefficient buffer reuse and many redundant copies between CPU and GPU. We're working closely with the ORT team to fix these issues though!

Odd, the links seem to work for me. What error do you see? Can you try on a different network (e.g., mobile)?

We’ve put out a ton of demos that use much smaller models (10-60 MB), including:

- (44MB) In-browser background removal: https://huggingface.co/spaces/Xenova/remove-background-web. (We also put out a WebGPU version: https://huggingface.co/spaces/Xenova/remove-background-webgp...).

- (51MB) Whisper Web for automatic speech recognition: https://huggingface.co/spaces/Xenova/whisper-web (just select the quantized version in settings).

- (28MB) Depth Anything Web for monocular depth estimation: https://huggingface.co/spaces/Xenova/depth-anything-web

- (14MB) Segment Anything Web for image segmentation: https://huggingface.co/spaces/Xenova/segment-anything-web

- (20MB) Doodle Dash, an ML-powered sketch detection game: https://huggingface.co/spaces/Xenova/doodle-dash

… and many many more! Check out the Transformers.js demos collection for some others: https://huggingface.co/collections/Xenova/transformersjs-dem....

Models are cached on a per-domain basis (using the Web Cache API), meaning you don’t need to re-download the model on every page load. If you would like to persist the model across domains, you can create browser extensions with the library! :)

As for your last point, there are efforts underway, but nothing I can speak about yet!

The 8-bit quantized version of the RMBG-v1.4 model is ~45MB, which makes it perfect for in-browser usage (it even works on mobile)!

Link to model: https://huggingface.co/briaai/RMBG-1.4

Paper: https://arxiv.org/abs/2312.00752 Models: https://huggingface.co/state-spaces

Hi everyone, Joshua from Hugging Face (and the creator of Transformers.js) here.

Starting with embeddings, we hope to simplify and improve the developer experience when working with embeddings. Supabase already has great support for storage and retrieval of embeddings (thanks to pgvector) [0], so it feels like this collaboration was long overdue!

Open-source embedding models are both smaller and more performant [1] than closed-source alternatives, so it's quite surprising that 98% of Supabase applications currently use OpenAI's text-embedding-ada-002 [2]. Probably because it is just easier to access? Well... that changes today! You can also iterate extremely quickly: experiment with and choose the model that works best for you (no vendor lock-in)! In fact, since the article was written, a new leader has just appeared on top of the MTEB leaderboard [3].

I look forward to answering any questions you have!

[0] https://supabase.com/vector [1] https://huggingface.co/spaces/mteb/leaderboard [2] https://supabase.com/blog/hugging-face-supabase [3] https://huggingface.co/BAAI/bge-large-en

This web-app fixes the two main problems of OpenAI's tokenizer playground: (1) being capped at 50k characters, and (2) not supporting GPT-4/GPT-3.5 tokenizers.

Everything runs in-browser thanks to Transformers.js.

Demo: https://huggingface.co/spaces/Xenova/doodle-dash Source code: https://github.com/xenova/doodle-dash

Whisper Web is a web-app which allows you to run OpenAI's whisper models directly in your browser, with no need for a server. This comes with the release of Transformers.js v2.2.0, which now supports multilingual transcription and translation for over 100 different languages.

Demo URL: https://huggingface.co/spaces/Xenova/whisper-web

Haha very interesting! I assume it's because that type of image is only found on computer screens, so, the model thinks the grass "contributes to it's idea of what a computer screen is".

... and of course, the library only ports those models to the browser; if you train a better model, you can always convert it to the ONNX format, then use it with the library.

Once ONNX runtime releases their WebGPU backend, we will add support for it! :)

It should also be noted that browser support for it isn’t very high at the moment… so, unfortunately, we are stuck with WASM (CPU) for now.

Here is the full list of available models: https://huggingface.co/Xenova/transformers.js/tree/main/quan...

As I mentioned in another comment, the library just allows the models to be run in the browser. The models generally give the same outputs as if they were run with their PyTorch equivalents, so, the quality can (for the most part) be blamed on the original model.

Also, remember to play around with generation parameters. Some tasks like code completion and speech-to-text work best with greedy sampling (sample=false, top_k=0), while others like text generation work best with random sampling (sample=true, top_k>0)

Yes, there are some workarounds you can do to get it working in non-browser environments. I do aim to get a permanent solution, which will ideally work out-of-the-box for both browser and node/deno environments.

Some other users also reported the issue (which stems from a bug in onnxruntime-web), and we were able to get it working in these cases:

1. https://github.com/xenova/transformers.js/issues/4 2. https://github.com/xenova/transformers.js/issues/19

Currently the library only runs on the CPU (it's only a few weeks old). WebGPU support is planned though (which releases soon [1])

[1] https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzP...

As shown in the demo video (on GitHub [1], or Twitter [2]), you do get that result sometimes (with randomness)

Using greedy sampling (sample=false and top_k=0) you get "Bonjour, comment êtes-vous?", which appears to be a very direct translation.

As mentioned in one of my previous comments, these inaccuracies also occur in the PyTorch models, and so, it's not the library's fault :')

[1] https://github.com/xenova/transformers.js [2] https://twitter.com/xenovacom/status/1628895478749315073

Right - currently, everything runs using WASM (32-bit, with 64-bit coming soon [1,2]), and I plan to add support for WebGPU soon!

(WebGPU is the successor to WebGL, which is coming out in April 2023 [3])

[1] https://github.com/WebAssembly/memory64/issues/36#issuecomme... [2] https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzP... [3] https://github.com/microsoft/onnxruntime/issues/11695#issuec...

Hi! Creator of the library here. If you change the generation parameters to be greedy (i.e., sample=no and top_k=0), you will get "Bonjour, comment êtes-vous?"

The top_k and sample generation parameters are just there to show that they are supported :), and is sometimes useful for the other tasks (like text generation w/gpt2, to get more variety)

Hi everyone! Creator of Transformers.js here :) ...

Thanks so much to everyone for sharing! It's awesome to see the positive feedback from the community. As you'll see from the demo, everything runs inside the browser!

As of 2023/03/16, the library supports BERT, ALBERT, DistilBERT, T5, T5v1.1, FLAN-T5, GPT2, BART, CodeGen, Whisper, CLIP, Vision Transformer, and VisionEncoderDecoder models, for a variety of tasks including: masked language modelling, text classification, text-to-text generation, translation, summarization, question answering, text generation, automatic speech recognition, image classification, zero-shot image classification, and image-to-text. Of course, we plan to add many more models and tasks in the near future!

Try out some of the other models/tasks from the "Task" dropdown (like the code-completion or speech-to-text demos).

---

To respond to some comments about poor translation/generation quality, many of the models are actually quite old (e.g., T5 is from 2020)... and if you run the same prompt through the PyTorch version of the model, you will get similar outputs. The purpose of the library/project is to bring these models to the browser; we didn't train the models, so, poor quality can (mostly) be blamed on the original model.

Also, be sure to play around with the generation parameters... as with many LLMs, generation parameters matter a lot.

---

If you want to keep up-to-date with the development, check us out on twitter: https://twitter.com/xenovacom :)

Hi there! Creator of Transformers.js here :)

I think it's worth pointing out that the library just gets the models working in the browser. The correctness of the translation is dependent on the model itself.

If you run the model using HuggingFace's python library, you will also get the same results (I've tested it, since, I wasn't too happy with those default translations and generations).

With regards to the text generation output, this is also similar to what you will get from the PyTorch model. Check out this blog post from HuggingFace themselves which discusses this: https://huggingface.co/blog/how-to-generate.