tmzt

A prehensile-tailed monkey? Watch them perform simple tasks with three useful appendages. (Really five)

The 40-bit hashed address seemed familiar, then I realized that I (and Claude) had used a similar concept in a VM we built for the OS I've been building. It's a capability and typing system that enables import boundaries to be enforced and third-party packages are unable to perform security sensative operations without being specially signed. It's also a "distribution of one" concept as the user builds their own UI to their specification personalising the OS.

I went ahead and forked this and added a new remote INFER protocol. You can see it at https://github.com/tmzt/ds4 if interested.

This looks useful for somebody with a 16-32GB Mac Mini interested in running larger MoE models.

I've been working on a mesh environment that relies on an explicit prefix hash in the request and enables constructing a new session with a cached pre-filled system prompt specific KV-cache beyond what OpenAI-compatible APIs offer. Can you see a feature like that being supported?

Awesome. I'd love to see this evolve into something, maybe a Matrix server of our own.

For now, I've been hanging out here as tmztmobile: https://matrix.to/#/#pocket-bootloaders:samcday.com

Updated and added links to gsmarena, thanks for that.

Thanks. I will.

I pushed all the intermediate branches as well.

So you've heard of USB?

You can with the newer instances that suport nested VM. There was a recent story about this here https://news.ycombinator.com/item?id=48556561.

Personally, I'm repurposing an older HTC phone with slide-out keyboard (the Speedy) with a new Linux port.

Do you see much of a difference between started Chromium instances with the same configuration in terms of the contents of allocated memory? Are they deterministic?

If not, could you template the memory and apply runtime patches (like timers or other initialized values) before releasing the process to run?

Would forcing the isolates to allocate memory better help at all, such as reducing fragmentation making your 2MB page sizes more effective?

I've been working on a mesh network for private AI models running remotely, controlled by mobile devices (smartphones, tablets, etc.). The mesh is constructed like a piconet, a few devices controlled by a single individual, layered on top of the internet.

How does it support semi-connected devices, intermittent connection failures, etc?

I thought they were itenerant troubadours.

Sounds a bit like Transmeta Crusoe.

Same. I switched my efforts to a larger Gemma 4 MoE model (26B-A4B) and llama.cpp and started getting meaningful results. I also implemented subagents for querying, determining which object/action to execute, and composing a short title. This is all running on an M4 in approximately 16 gb of ram. Also using Google's native tool calling channels.

It's basically restricting what logits are allowed when sampling the model to conform with the JSON (or whatever) shape. It can also cause the model to get "confused" though and doesn't always result in the output you want.

Doing the same for Apple M-series with fused wgsl shaders specifically targeting Qwen3/3.5.

My effort is called shady-thinker and is on github at github.com/tmzt/shady-thinker.

This was inspired in part by Antirez's earlier work with C kernels as well as other efforts to support in-browser LLMs. I've adapted them to Rust and the wgpu library.

Gemma 4 is also the next likely target (with the MTP work) as I'm experimenting with local AI agents.

I'd love to see what you've done to improve prefill and decode even if its not directly applicable.

One difference, I'm using MLX and GPTQ 4bit quants including AutoRound with safetensors as my shader pipeline is pretty much fixed for each model, ggml just adds unnecessary complexity.

I'm working on this too. I'm building a distributed environment where compute and GPU resources can be on one system, display on another, organized into a type of piconet. So far, I'm working with offline/local AI though running into limitations with it. The goal is to allow for a user to customize the environment with personalized cards powered by open schema databases. The same UI works on mobile devices, tablets, desktops, even TVs and IoT devices (ESP).

While it's true that early Linux ARM devices where embedded and generally only supported a single configuration, they didn't actually use devicetree.

Originally, embedded Linux ARM devices used a board file with a platform bus and hard-coded device metadata. The bootloader had to pass a machine id which told the kernel which hardware you were running on and which board file to use.

You can see remenants of this in the kernel still, though it's quickly being removed. I'm actually working on a hybrid kernel with the goal of bringing modern Linux support (on an lts branch) to old MSM7x300 devices, like the Evo 4G Shift I intent to use a tmux console/cyberdeck.

On another note, ACPI/UEFI doesn't always give you a clean abstract surface to work with either. ACPI is notorious for building in OS checks into it's compiled bytecode to the point that Linux often lies to it about what OS is running.

Has anybody thought about encoding AST tokens as LLM tokens, similar to how different words can have different meanings and that's reflected in their embedding?

Also working on something similar and using a dual LLM architecture (small router, larger deep thinker) with offline models, as well as determinisitic skills encoded as TSX.

It's evolved into a mesh-based operating system, gained it's own GPU-based AI library/runtime, and even molted and extended itself to ESP nodes.

Getting closer to a full release sometime in May. For now, pieces are released on my github.

If that type of scanning works and discovers these attacks, why not make it part of the publishing process?

Using LLM to target an FPGA would be interesting, generate the Verilog/VHDL.

is there a link to the project, the one in the comments won't open on mobile

If it helps I've been doing a lot of projects with TSX (via OCX) as a file format.

It works nicely for these things because it already supports typed parameters and can be treated as a a raw AST without JS engine.

Its also easy to manipulate with the same library.

Because one is for lawyers? It doesn't seem like a hard choice.

Same here. Then you see SOTA in a browser from Ex0byt, online 10x training (JIT-Lora), TurboQuant (Google), etc. Just saw KV prediction mentioned in this thread, so looking into that too.

I'm adapting all of this to Rust+WGPU with compute shaders if you want to follow along.

See this repo: https://github.com/tmzt/shady-thinker

Goal is Qwen3.5 27b on a Pixel 10 Pro running GrapheneOS.

Gemini Nano is supposedly doing it on device. It looks like something similar should work with Apple GPU and ANE.

Or a grinder and a camera. See CCC of years past.