josefcub

I would recommend against this. Check your local and EU laws -- Anything coming out of your IP address may end up being your responsibility if it turns out to be illegal.

Another thing to consider is that most people would likely rent it for the unified memory and AI usage. Speaking from experience, your M3 Ultra will run _hot_ while performing; you'll need to plan a good, cool environment that the machine lives in, and set the fans to always run via something like MacsFanControl or some other utility, since it won't engage the fan at a higher rate just because your GPU is pegged and generating all the heat.

The third thing is knowing how to balance out its memory usage. SSDs have a finite lifetime in terms of writes. Macs are _very_ good at not wearing out SSDs, but if your renter botches something in a config and you spend days or weeks swapping in and out of virtual memory, your machine will wear faster and harder.

Well, I ran a couple of experiments a couple years ago against a 10.7b SOLAR-based language model and MUDs. What I found is that dumping one into a MUD that had been built specifically for humans resulted in a lot of confusion that usually ended up with the model looping around in a circle looking for something or someone to interact with.

When I repeated the experiment with a MUD that I'd built by hand (A small American town) for the LLM's own limitations (Descriptions referenced things that I made sure existed, more common verbs existed for it to use on things, there was a map facility, and at least me to interact with on a second connection), I found the agent much more likely to take its time exploring, making up its own goals, and spending time traveling in the space just communicating with me in a roleplaying context.

It was an interesting time; I wasn't sure what I was expecting it to do after the first experiment, but it seemed to really jump into the second one and kept playing until I terminated the experiment.

If I were going to do it a third time, I'd probably create objects and give a modern agent fetch quests and other goals, and see how well it independently can handle that.

I've never used such a thing. If the intent is to get the LLM to not perform reasoning before it answers, most of the APIs have a toggle that lets you minimize or stop thinking.

I'd really like to understand why you'd use "Answer now" though; what's your use-case?

Hermes Agent is really decent.

Don't rethink your entire company, offer your customers choice instead. Some may prefer OpenClaw and others might be better served with Hermes Agent or even a third type. If you can find the best use-cases for each, you can help customers decide what they need when they buy your service and get all of the customers instead of a subset.

I'd started using DOS-based machines in the 80s, and ended up with a copy of WordStar 3.3. I used it so much as a kid that I still have the WordStar keybindings stuck in my muscle memory.

There's one CLI text editor left with WordStar keybindings used natively: Joe's Own Editor. If you're like me and enjoy WordStar's key combinations and the editing paradigm, JOE is a good text editor to use on the command line.

Brave's "Leo" AI is configurable enough to specify local endpoints for processing, instead of going wherever they want it to go. I've set it up to use my own systems, and it works just fine like that.

If you have a beefy enough device, then yes this can be done on-device.

I've got 256GB of RAM on a Mac Studio M3 Ultra. Other posters are right: The M3 Ultra's prefill is super slow with really large models, 3-5 minutes while it digests the new additions to its context before it continues. On my heavy RAM model, I _can_ run 400b-500b models at Q2, and up to about 750b models at Q1, but the wait isn't the worst part.

Lower quants like that affect its output, making it less capable overall and letting it easily forget things.

Here's what I'd do with 96GB of RAM: Run Qwen 3.6 35b-a3b at Q8 for coding/agentic tasks. You'll get around 70tokens generated per second, the prefill is lightning fast in comparison, and you'll get a lot of work done. Qwen 3.6 27b is out now too, and I'm getting 17tok/sec token generation with a slower prefill.

The upshot is that you'll still have 20-40GB of RAM left for your workstation and development loads. Running Qwen 3.6 35b or 27b at Q8 quantization, the model at 128k context uses about 40GB of RAM; my OS and application load uses 20-30GB most of the time, for a total of 60-70. That's plenty of room in memory for you to work _and_ run inference.

You _may_ end up getting Deepseek 4 Flash running, but it'll be a lower quantization like Q2 or Q3, making it kind of dumb in comparison. And you may not have enough memory left over for any appreciable amount of context. Working with today's reasoning models needs context for it to generate and give out good answers. Doubly so for agentic/coding tasks.

My google-fu is failing me at the moment to cite sources, but here's an example ~/.config/crush/crush.json file (based on my own) showing the options to remove telemetry and provider auto updates, and the connection info to connect to a localhost model on an OpenAI-compatible endpoint:

{ "$schema": "https://charm.land/crush.json", "options": { "disable_provider_auto_update": true, "disable_metrics": true }, "providers": { "ollama": { "name": "Local Models", "base_url": "http://localhost:11434/v1", "api_key": "nunya", "type": "openai-compat", "models": [ { "name": "Qwen 3.5 Local", "id": "qwen-3.5-35b-planning", "cost_per_1m_in": 0.01, "cost_per_1m_out": 0.01, "context_window": 131072, "think": true, "default_max_tokens": 5120, "supports_attachments": true } ] } } }

...or not, thanks to formatting. I can't even search for help formatting this text box, because of HN's nature haha

Try charmbracelet's crush, found here:

https://github.com/charmbracelet/crush

Crush is pretty new, but getting better all the time. It's written in Go, so no node hijinks to get it working. It works fine with my ollama or llama-server localhost endpoints, and I've used it to make up a couple of internal projects without any issues.

It does have internal telemetry and such (including updating its list of external models it can use) that can be turned off in the crush.json configuration file.

If you're on a Mac, you can install via homebrew or use the more traditional route via Github.

I am! I moved from a shoebox Linux workstation with 32MB of RAM and a 12GB RTX 3060 to a 256GB M3 Ultra, mainly for unified memory.

I've only had it a couple of months, but so far it's proving its worth in the quality of LLM output, even quantized.

I generally run Qwen3-vl at 235b, at a Q4_K_M quantization level so that it fits, and it leaves me plenty of RAM for workstation tasks while delivering tokens at around 30tok/s

The smaller Qwen3 models (like qwen3-coder) I use in tandem, of course they run much faster and I tend to run them at higher quants up to Q8 for quality purposes.

The gigantic RAM's biggest boon, I've found, is letting me run the models with full context allocated, which lets me hand them larger and more complicated things than I could before. This alone makes the money I spent worth it, IMO.

I did manage to get glm-4.7 (a 358b model) running at a Q3 quantization level; it's delivery is adequate quality-wise, although it delivers at 15tok/s, though I did have to cut down to only 128k context to leave me enough room for the desktop.

If you get something this big, it's a powerhouse, but not nearly as much of a powerhouse as a dedicated nVidia GPU rig. The point is to be able to run them _adequately_, not at production speeds, to get your work done. I found price/performance/energy usage to be compelling at this level and I am very satisfied.

I think this is fun! I ended up with 92/102, because a few were non-obvious. If I had a critique it's that the AI entries were repeated on multiple occasions, making the decision between them trivial.

If you can find some public domain literature from the 20th century, it would be a much harder game.

I'd listen to that on the radio! LOL

2.3L 4-cylinder. Good enough for the light truck tasks I usually end up having. It's the last year of the first generation Rangers. I had a 1988 model with a 4 cylinder when I was young, and -that- one trucked along through all the youthful abuse I threw at it. I'm really looking forward to enjoying this one too.

I'm going to buck trends and get my hands dirty -- The oil needs changed and the O2 sensor needs replaced on an old (1992) Ford Ranger pickup truck I recently acquired. Bought it for a song, and it's definitely not a looker but it's still going strong (especially after the small age-related repairs I'm doing).