I would love to see someone build it as some kind of an SDK. App builders could use it as a local LLM plugin when dealing with data involving sensitive information.
It's usually too much when an app asks someone to setup a local LLM but this I believe could solve that problem?