docheinestages

people who will have the technical know-how to facilitate custom CSS

I think anyone with access to an LLM doesn't necessarily need the technical know-how. They can just ask as an extra step for a design check including text contrast.

I feel low-contrast dark mode is precisely the aesthetic

Agreed, and I like it too, if legible.

True, maybe we can call the generated SVG something else than art.

I think a more fundamental test is SVG art creation in general. Perhaps a pipeline to take any image, caption it, ask the LLM for an SVG, rasterize to an image, and finally either use a deterministic visual similarity check or ask another LLM to be the judge and score how close the SVG is to the original image.

Is this really not a mostly “solved” problem using assistive browser (or OS-level) ... ... performatively sacrifice aesthetics ...

One doesn't need to sacrifice aesthetics to make the design legible to an average user.

At what point is it enough?

It depends on who your audience is. The more people you want to reach, the more you need to cater to everyone's needs. There are fundamental anti-patterns to avoid like writing with a dark font on a dark background.

The landing page doesn't show a link to your GitHub. Despite mentioning "self-hostable", I genuinely thought it was closed-sourced. I doubt first-time users will trust a random tool and grant it database-level access with a curl command.

I loved the text contrast on the landing page. Grey text on a dark background.

Any estimates of the performance (prompt processing and decoding tokens/s) on consumer hardware like Macbook Pro M-series?

That's why I believe the baked weights should represent a generic model that is good at general thinking and tool calling. The rest can be augmented by the OS.

I got the same vibes. I really wish YCombinator would mentor startups on design. Unfortunately, most of the landing pages in their startup directory look like raw AI designs.

I got the same impression looking at your website and demo video. They both came across as generic AI-made content.

I think AI will evolve the same way computers did. We're somewhere in the 80s-90s timeline of the evolution. My prediction is that on-device models will have excellent tool-calling, reasoning, and general skills, but the domain-specific knowledge will be retrieved on-demand from vendors like Google. Rather than downloading models, each device will have a hardware component with weights baked into silicon for maximum efficiency.

True. It was more of an open letter, with hopes that the Qwen team sees the comments in this thread.

Qwen has set an excellent track record for architecting and releasing open-weight models that consumer-grade devices can run. What is needed the most right now is something similar to Bonsai 27B, with a modest memory footprint, but faster and more capable. On-device models can make up for intelligence by being faster, thinking longer, or doing more quick iteration rounds.

The cloud era already ran this experiment: proprietary APIs plus data gravity made exit punitive. The repatriation wave is the receipt.

Some sentences smell a lot like AI.

Any VC-backed company will eventually pivot.

I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing details. That needs effort, with or without AI.

Oh, I just realized Microsoft removed it a couple weeks ago. I had the link in my bookmarks. The model is available on HuggingFace [1].

[1] https://huggingface.co/models?sort=trending&search=fastconte...

Look into FastContext by Microsoft. Not extraordinary but specifically designed for paired usage with a larger LLM to save tokens [1].

[1] https://github.com/microsoft/fastcontext

It really depends on which angle you look at it. Is it purely to meet a business goal? Or is it also for personal growth? I think it's a mix of both, but for me it's always important that I engage mentally with the process, learn something, and solve puzzles, even if that involves letting the AI take care of the coding, which is an abstraction. You could still code and not think creatively.

I think the most secure setup, though not so convenient for the average user, is a separate machine with QEMU/KVM. The machine should be isolated adequately, such that even if compromised, it shouldn't be able to cause damage or gain access to other machines. Additionally, a proxy server on your machine or elsewhere could hide sensitive credentials. A helper binary on your computer would then control spawning new disposable VMs with premade images and your SSH key. The images can be lightweight or the desktop version with a remote VNC.

I'm on a 48 GB M5 Macbook Pro. I use 4-bit quants with a context window of 16-32k. I tried Qwen 3.6 27B, but I can only get around 10 tokens per second, but it's painfully slow, and it often fails during `write_file` tool calls, even with Qwen Code.

I'm looking for something that runs on an M5 Macbook Pro with 48 GB of unified memory.

I've been trying various harnesses like Pi, OpenCode, Qwen Code, and Nanocoder. A common problem I keep running into is failed tool calls, regardless of the model. What is the best harness and on-device model combination right now?

I think this person is exhibiting symptoms of prolonged conversation sessions with LLMs. Maybe this is how they talk/shout/scream at the AI, which has no emotions, and gladly accepts this tone of speech.

Do you mind sharing the code?

I think it's good etiquette to ask the user if they want to download the model or any assets larger than the average page size, in case they have limited bandwidth.

Every browser should have an embedding API to avoid having to worry about the plumbing and data transfer.

I think humans don't behave the same as bots when playing this game.

Why not! And free hot showers during the summer. Just looking at the progress made since 2023, I don't think the LLM architecture we have today is the most efficient. We need creative game developers to start making LLMs.

What we really need is a breakthrough in inference or LLM architecture to allow running GLM-5.2-level models at the size of Qwen 3.6 27b or smaller on consumer devices like a 48GB Macbook Pro, and at least at 100 tokens/second. My hypothesis is that a smaller, less capable but faster model paired with a good harness can run for longer and brute force its way out to solve problems that the bigger models can one-shot.