tbocek

Just today, I tried to run docker compose on a remote host via podman-docker on Fedora (Asahi). I ran into all sorts of buildx issues, and the easiest fix for me was to remove podman and install docker instead.

I tried working through it with Claude, but after a few failed attempts I gave up. I'd like to use podman, but the docker compose + buildx compatibility gaps made it more trouble than it was worth for now. I'm definitely going to try it again.

That's a valid concern. I took a more constrained approach for web searches for exactly this reason. Instead of giving the LLM full browser control, I built a Firefox extension that only handles web search client-side.

When my local LLM (llama.cpp) needs to search, it opens DuckDuckGo in a new window, loads the result pages in tabs, extracts content with Readability.js, and feeds it back. You stay in the loop - can see what's loading, solve captchas if needed. Less autonomous than Comet/Playwright, with a narrower use-case, but also less risk.

Its still a prototype though: https://github.com/tbocek/llm-local-web-search

Thx, I cannot change my previous message, the QH is set to private, as me and my students are working on it. It will be made public next month. The qh:// protocol, we have just started, and we only focused on simplifying the header as an alternative to HPACK / QPACK. Currently we are collecting headers for identifying what should go into our static table for lookup, and for benchmarking. For simplicity, we decided against dynamic tables or huffman-style encoding/decoding. Just a table with often used header/values, the rest is binary encoded with no further compression.

UDP-based protocols are well suited for P2P, since hole punching is straightforward if you have predictable port mapping, you cannot disallow it. In that spirit, we are currently exploring this with:

https://github.com/tbocek/qotp and https://github.com/qh-project/qh

The main idea is to have a simple encryption (ed25519/chacha20+poly1305) for encryption in the transport layer, on top of that then qh, where certs are use for signing content.

With out of band key exchange, you can establish a connection after you successfully punched a hole.

However, its not QUIC compatible in any way (https://xkcd.com/927)

This is probably due to this: https://github.com/ggml-org/llama.cpp/issues/12637. This GitHub issue is about interleaved sliding window attention (iSWA) not available in llama.cpp for Gemma 3. This could reduce the memory requirements a lot. They mentioned for a certain scenario, going from 62GB to 10GB.

Here is mine, I have a docker compose file locally, and this deploy.sh script deploy to my remote machine. That also means that my remote machine is building the image. And I have not found a good solution for secrets/env files yet:

  #!/usr/bin/env bash
  
  export DOCKER_HOST="ssh://username@host:port"
  docker compose up -d --build

Just did a test with https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-32..., with the question "what happened at Tiananmen Square?", and here are parts of the thinking process:

  ...I also remember something about the "Tank Man" image, where a lone protester stood in front of a line of tanks. That image became iconic, symbolizing resistance against oppression. But I'm not sure what happened to that person or if they survived.

  After the crackdown, the government censored information about the event. So, within China, it's not openly discussed, and younger people might not know much about it because it's not taught in schools. But outside of China, it's a significant event in modern history, highlighting the conflict between authoritarian rule and the desire for democracy...

There is a difference between chat and code completion. While with chat, you can use localhost with llama.cpp, but code completion you cannot do that: https://github.com/zed-industries/zed/issues/12519.

The config for chat, you can do:

  "language_models": {
    "openai": {
      "version": "1",
      "api_url": "http://localhost:8080",
      "low_speed_timeout_in_seconds": 120,
      "available_models": [
        {
          "provider": "openai",
          "name": "Qwen2.5-Coder-7B-Instruct-Q8_0.gguf",
          "display_name": "llama.cpp",
          "max_tokens": 131072
        }
      ]
    }
  },

I had the same issue, so I implemented the following workaround: https://github.com/tbocek/dvorak. This remaps on Linux CTRL-I back to CTRL-C when using a Dvorak keyboard layout.