xyc

I'm running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok/s for 20B model and 12 tok/s for 120B model. I'd say it's pretty usable.

For TypeScript you can refer to https://github.com/modelcontextprotocol/typescript-sdk/blob/...

There isn't much documentation available right now but you can ask coding agent eg. Claude Code to generate an example.

Great to see more local AI tools supporting MCP! Recently I've also added MCP support to recurse.chat. When running locally (LLaMA.cpp and Ollama) it still needs to catch up in terms of tool calling capabilities (for example tool call accuracy / parallel tool calls) compared to the well known providers but it's starting to get pretty usable.

It's a protocol that doesn't dictate how you are calling the tool. You can use in-memory transport without needing to spin up a server. Your tool can just be a function, but with the flexibility of serving to other clients.

recurse.chat + M2 max Mac

I recently discovered toolhive which is pretty handy too https://github.com/stacklok/toolhive

If you are on a Mac, give https://recurse.chat/ a try. As simple as download the model and start chatting. Just added the new multimodal support in LLaMA.cpp.

Actually this is a good way to find product ideas. I placed a query in Grok to find posts about what people want, similar to this. Then it performs multiple searches on X including embedding search, and suggested people want stuff like tamagotchi, ICQ etc. back.

It seems that this is possibly not necessary, since LLaMA.cpp already integrates Jinja with CPP implementation (through minja)

Check out https://recurse.chat/

The fact that there's no alternative implementation of SQLite also seems to play a part in preventing standardization of WebSQL.

https://www.w3.org/TR/webdatabase/

"The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path."

If anyone on macOS wants to use llama.cpp with ease, check out https://recurse.chat/. Supports importing ChatGPT history & continue chats offline using llama.cpp. Built this so I can use local AI as a daily driver.

You can get a release binary from https://github.com/ggerganov/llama.cpp/releases too.

Made tool use work! check out demo here: https://x.com/chxy/status/1861684254297727299

sharing the messy code here just for funsies: https://gist.github.com/xyc/274394031b41ac7e8d7d3aa7f4f7bed9

Just tried out the puppeteer server example if anyone is interested in seeing a demo: https://x.com/chxy/status/1861302909402861905. (Todo: add tool use - prompt would be like "go to this website and screenshot")

I appreciate the design which left the implementation of servers to the community which doesn't lock you into any particular implementation, as the protocol seems to be aiming to primarily solve the RPC layer.

One major value add of MCP I think is a capability extension to a vast amount of AI apps.

^ asked the question in the discussion: https://github.com/modelcontextprotocol/specification/discus...

Thanks for the pointers! Will do. I've fired up https://github.com/modelcontextprotocol/inspector and the code looks helpful too.

I'm looking at integrating MCP with desktop app. The spec (https://spec.modelcontextprotocol.io/specification/basic/tra...) mentions "Clients SHOULD support stdio whenever possible.". The server examples seem to be mostly stdio as well. In the context of a sandboxed desktop app, it's often not practical to launch a server as subprocess because:

- sandbox restrictions of executing binaries

- needing to bundle binary leads to a larger installation size

Would it be reasonable to relax this restriction and provide both SSE/stdio for the default server examples?

Superb work and super promising! I had wished for a protocol like this.

Is there a recommended resource for building MCP client? From what I've seen it just mentions Claude desktop & co are clients. SDK readme seems to cover it a bit but some examples could be great.

With Claude you barely had to learn the language this days as you just need to prompt, but SQLite column is an interesting idea.

If you are interested in no config setup for local LLM, give https://recurse.chat/ a try (I'm the dev). The app is designed to be self-contained and as simple as you can imagine.

Shameless plug: If you are on a Mac, check out RecurseChat: https://recurse.chat/

A few outstanding features:

- Fast: Import ChatGPT history, loads thousands of conversations at once.

- Floating Chat: spotlight / ChatGPT desktop app like floating window.

- Customization: You can add any OpenAI compatible API (including X.ai) as a model, and just edit the url/model id

- Chat with files: Not as complete as complete as a RAG solution for now, but we feature simplicity. Basically you can drag and drop PDF files onto a session or add files/folders to a model (like custom GPT) to start chatting.

- And, yes we support light mode! And several light mode code themes as well

Regarding chat history, I've been thinking that people should have ownership over chat history. We are migrating the chats towards SQLite, so your data is going to be a timeless format like files* - SQLite has long term support through year 2050. https://www.sqlite.org/lts.html

* See [File over app](https://stephango.com/file-over-app).

Give https://recurse.chat/ a try - I'm the developer. One particular advantage over alternative apps is importing ChatGPT history and speed of the app, including full-text search. You can import your thousands of conversations and every chat loads instantly.

We also recently added floating chat feature. Check out the demo: https://x.com/recursechat/status/1846309980091330815

what's your experience with sqlite-vec? I'm considering using sqlite-vec in addition to/or replace qdrant vector db for a project (recurse.chat), since I'm moving all the data to sqlite. I love everything SQLite so far, but haven't got to try out sqlite-vec yet.

Neat. I automatically upvote / like when one builds <not-a-game> with a game engine.

UX of the website is honestly underrated, and I think if every website has an HN/Craigslist mode it'd be so awesome. I just wished for additional functionalities and a brief search seems to indicate people resort to external websites for notifications.

Most likely someone will make desktop/mobile clients for popular websites with minimalist aesthetics. Last time I tried a couple but didn't find it useful.

The website is great, fast and slick. but sometimes I wished there were notifications when I needed it. Of course a mute everything option would be handy as well.

Like for this post it'd be faster if I can click a button to see the responses instead of go to my submissions.

Things like grab some markdown text and ask to make a pip/npm install one liner, or quick js scripts to paste in the console (which I didn't bother to open an editor), a fun use case was random drawing some lucky winners for the app giveaway from reddit usernames. Mostly it's converting unstructured text to short/one-liner executable scripts & doesn't require much intelligence. For more complex automation/scripts that I'll save for later, I do resort to providers (cursor w sonnet 3.5 mostly).