sipjca

It's not that much extra work to compile, the extra work comes from the maintenance and feature requests. By not shipping the binary directly I am defending my time until other contributors want to step up and maintain things. I am one person with limited time and I don't want to spend all of it in front of a computer

Yes you use them through a wheel. If you have specific questions on packaging and how to use things lets move it over to the discussions/issues in the repo itself so it can be more broadly accessible to more people and we can make the packaging of the library as useful as possible

we have wtype and ydotool support as well in Handy, but it's not exactly a great out of the box experience always, but some of it is on me for poor linux packaging

Right now I intend to maintain this as a library. The examples are just that, examples for programmers/agents. If someone in the community wants to step up to maintaining release binaries I will gladly have that support, it's just impossible to do as a sole maintainer

Wow, it's amazing to hear that even though I released this so recently people are already using it properly! Thanks! Please let me know any issues you run into

yes

word boosting will probably come on a much longer time horizon, but streaming is here!

I'm really hoping someone either contributes a good server example to the codebase (and is willing to help with issues) or use transcribe.cpp or the bindings to create a robust server in another language :) would be happy to link it from the main project directly as well

I’m on a train right now but off the top of my head the audio pipeline may have to be modified slightly to emit partial text segments as they come in from the transcription engine. And then calling the appropriate paste method the user has in their settings.

It may be easier than expected in some way since we already emit events for the live overlay, so it could be as small as a function call, but I don’t know the code path well enough from memory and what complexities it has. Probably with the Tauri context and a bit of other mess we have as this bit of code has gone through a lot of pain

Eventually I would like to be more fully drop in compatible, right now some feature support is a bit sparse. And whisper has so much work done to it over the years so it’s hard to support every possible thing. Right now it’s a more bog standard implementation than anything special. Right now stabilizing the core header is probably among the primary goal, but if people want to contribute model specific things im happy to review test and pull in. Whisper is a good case for this as there is a header extension already so it’s easier

For sure, it was not initially a target because I didn’t need it for Handy but I do understand the importance in the broader context

Testers by far as the most needed thing, I do maintain a list of per platform people who help to test so if you drop a GitHub username (or email me) I will add you to the list and ping for help

Basically the biggest blocker is me being the sole maintainer and reviewer at the moment and it just ends up taking a lot of time for the scale of the project. Which is why it moves slow and features typically are much slower than someone can vibe code. I know each added feature inevitably has bugs so I try to be careful with them.

But also Linux has historically been a minefield, fixing something for someone breaks for someone else so yeah testers really needed. Or anyone with deeper Linux DE knowledge than I have. I’m much more accustomed to server based Linux distros

Every model with open weights has some code which can be used to inference it. So we download the published weights and run against inference library they suggest, be it transformers, Nemo, etc

Largely this is out of scope for the library, mainly because I’m not aware of many models supporting this. but if there are models which support this would be happy to support

Yep, could do simple things like literal regex or all the way up to LLM cleanup, tons of options

Yeah I’m working on it, Linux is a big pain point especially Wayland

Once things are more or less ironed out on MacOS and Windows a lot of attention will be turned towards Linux

I know a lot of Linux PRs are open it just takes me so long to get around and test them. And often multiple different implementations trying to fix similar issues which is a lot of overhead sometimes

TensorRT and CUDA is effectively the same speed as CPU for the speech to text models I was testing via ONNX at a huge binary bloat penalty. WGPU is hard to ship and also equivalent speed or slower. This may not be the case for LLM or other models but the runtimes did not seem well supported for what I needed to do. ONNX is incredibly well optimized for CPU, best in class even, but the other execution providers at least for STT seemed lacking.

I did this investigation before creating transcribe.cpp it would have been much more convenient and save me literal months of work. Happy to share the repo and binaries produced as well, but it was mostly throw away work to profile how to ship accelerated ONNX in Handy.

Not in the library itself, it’s pure inference. Some models have this trained out of them anyhow. Otherwise this is a post processing task which is not really inference

Out of the box no probably not, but if people are interested there’s probably ways forward

You can fairly easily modify [Handy](https://handy.computer) to do this if you want

I’m planning on having it as a first class feature of the app too just too many other issues to work on first

Parakeet unified for me no longer does this and it’s also a streaming transcription model!

But the answer largely depends on you, the languages you speak, and personal preference. Whisper is still excellent and supported in transcribe.cpp

Cohere Transcribe is also excellent, but many of the new models are as well

Contributions are always welcome! There’s a WIP diarization PR rn, and after it’s merged would love to have support if it fits well into the interface. And if not would love to figure out a good interface for it

Ha amazing, love to hear it

Yep, but I am in the process of also porting NVIDIAs Sortformer for multi speaker diarization as well :)

I’m not sure how many specific models will be supported as the library is more focused on transcription specifically. But the models which support diarization natively must be supported I think. And parakeet multitalker was the primary driving force for this change

Thanks! What an excellent question, I’m not sure I have a good answer. I kind of became an open source maintainer by accident as Handy became popular

Certainly I am very lucky that quite a few people donate to Handy, and also some people and organizations who sponsor the work I do

To be honest I just love contributing to open source and wish to continue to do so. So anyone who supports this is good to me. Organizations which believe in OSS and push it forward are typically most aligned with me

Of course you can always email me (contact@handy.computer) and we can discuss in more detail

Yes, I’ve put a PR up on pypi for extra storage for CUDA but it has not been accepted yet afaik

If there’s any issues or improvements on the bindings I would love help to make the DX the best it can be

It very much depends on the hardware! An M4 max is being compared against a Ryzen 4750U with an integrated GPU!

The M4 max has probably 10x the compute and memory bandwidth hahaha

Yep the latest version has support! Virtually all of the SOTA open models are supported by Handy including the streaming ones like

Nemotron Streaming

Parakeet Unified

Voxtral Mini Realtime

If something you want is not supported, open an issue on transcribe.cpp!

Thanks :)

More or less yes, for whisper.cpp, just trying to make local transcription more accessible to anyone building an app, etc

Hey, yep author and maintainer here! Certainly sponsors help and the wonderful community who donates to Handy as well! Mozilla AI was very helpful in getting this work off the ground. It was a pipe dream for me to build for Handy and they helped to sponsor me so I could make time to take this project seriously and get a v0.1.0 release out the door

I agree this should be everywhere and I hope to distribute libtranscribe some day properly so it is more a system library! It will take time to stabilize but I think we can get there

Hey! It’s actually in progress right now, probably will come this week :)