edg5000

I always let GPT 5.5/5.6 sol reach compatction. I'm only offered a 258K window, but I think once they release 1M to normal plan users, it will be usable across the whole 1M. At least with Opus I could use the whole 1M. I'd argue that the more context I use, the better performance gets. It has more info already available. I don't notice degradation.

I initially thought that getting agents to work for longer and in large groups was the future, but I'm increasingly thinking that, at least for engineering, just one thread makes more sense. The agent pulls things into context as needed. One thing that I've been experimenting with is also letting the agent remove things from context, such as files. But just adding to the context and compacting when it's full seems like it might beat a lot of more advanced options. Because the model is good, it knows what to put in the summary; just enough for the model to be able to rebuild the context from that seed (e.g. pulling in relevant files/data into the context).

I generally use: - Overbuilt: Designed to handle more of the same. Too strong. - Overengineered: Complexity not justified by the requirements.

Sometimes overbuilding (e.g. using really thick wood on a treehouse) means you can lower complexity (less bracing needed) at the cost of being more wasteful with material. That is often a great tradeoff.

I could not disagree more. You must have used the better models by now. Even if you only lightly use them for a question from time to time, it must have become clear that these things are pretty damn good and aren't going away?

When doing hard things I need to be 100% in the loop, but I don't have to do anything menial anymore. Imagine being in some CIA control room with 10 emplyees, and you're directing the team. You're still on top of things, just not bogged down with bulky minutiae.

With a coding plan, there is no need to use lower quality models right (GPT 5.6, Fable 5). Only with some kind of batch job (processing PDFs) I could imagine not having enough usage with a 100 or 200 USD coding plan. It sounds like you're on API? I'm dying to know why, since it seems quite common, yet it's about ten times the cost. Sounds like a terrible deal. What am I missing?

Yes, but you could have an uncached dynamic tail, this would not kill the cache though. And you could still periodically kill the cache, still keeping the same prefix every N requests. But it's questionable what all this extra complexity delivers.

I found that now that the models are better, at least Codex compaction with GPT 5.5+ xhigh is almost unnoticible. So the answer is: just let the context fill up, don't prune much, the model is smart enough. When you're out of space, have the model summarize and continue from that point. A smart model will know the information pieces it needs to re-read after compaction; it won't make incorrect guessed when some details got lost.

It's a very interesting technology, so it leaves some shockwaves while it's impacting the world. Over time this will be absorbed and the insanity will die out, but it's quite transformative, just like cars and the internet were (to name a few).

That makes sense.

Yeah, good point. The simpler solution eventually wins. But I do think a GUI makes more sense though, that was the reason I went my own way, but I think you're right that keeping it barebones is the way.

Great speeds. So a small model on a lot of slow ram is fast. Now I wonder how larger models run on that thing.

I do about 10 google search queries for every 1 opus/gpt prompt. For google, I don't actually open pages anymore 9 out 10 times; I rely on the AI summary. It's fast and accurate; the trick is that you learn where the boundary is of what you can ask it. Querying information the small model is great at.

Then there might be slow, batch tasks. I can see myself getting 1T of slow RAM one day (in a few years?) and having a slow onsite GLM5.2 doing batch jobs that would be wasteful of my subscription limits, plus sensitive but boring things, such as bookeeping and general admin.

I'd like to to read all my email and al quarterly reporting. But that would have to be a good local model, probably a model simmilar to whatever google search uses, which seems just correct unless you throw serious challenges at it.

I ask it to propose revisions, which I then review and provide detailed feedback on. It then works, and afterwards I review. I always commit beforehand. This works well for me.

I've experimented with completely novel tool call syntaxes combined with ordering and structuring the context differenly, and they seem to have a high tolerance for this stuff. But you always risk losing performance, probably highly unlikely to exceed stock performance across the board.

I spent 6 months frantically working fulltime on a harness, and when it was finished, I just stopped using it and went back to Codex CLI. I've also stopped using all the fancy features: subagents, planning mode. I've also stopped worrying about context, since I just let codex hit compaction. Somehow this doesn't seem to hurt much. Weird.

Their PCB service must be a loss leader, but their assembly service might be profitable. What's also a loss leader is their default configuration. Want .2mm vias? That's another 30$. Want to assemble both sides? More fees. So it gets more profitable for them not just with volume, but also with designs that require the add-on fees.

I like how they're using real numbers and theorizing around that. An interesting next step would to dive in deeper, to see up close where it's bottlenecking.

Even if it's uploaded once, it's still being ran through inference. It saves a bit of HTTP traffic I suppose.

None of them upload the whole repo, which is what this link claims it does. That's unheard of.

Wow, very interesting fact. If a unit is performing badly due to lack of equipment, they won't get the points to get the equipment they need. I wonder if that's a problem in practice. I'll read up on it, very interesting stuff.

Fun sidenote: On a recent Hotmail post on here, I saw a screenshot of hotmail V1 (way before M$ took over). They had a prominent button on their homepage about privacy.

It scores high on BenchCAD, that's interesting to see, I was wondering about how each model could handle this. Seems like they trained it on programmatic CAD specifically.

This comment is odd, not conventional — it reads suspiciously verbose, not consice.

We can look at openrouter GLM 5.2 prices I think to get a rough idea about the pure inference cost (with margin). They are probably running on simmilar hardware. Although scale helps Anthropic probably.

Because it'll bottleneck at the CPU? Or because 200 bB/s is overkill when running a model (that fits in memory)?

Those font tests are cool, he's created a unique way of rendering fonts, an interesting, playful look. And great that he's letting his son play with the computer.

Any good LLM will emit Python or other scripts to analyze or work with large files (e.g. GeoJSON) naturally when asked to work with a large file. So I don't see that as needing an explicit solution. The LLMs just figure it out.

Your argument is that it's simply because of dropping prices. Maybe that does explain the bulk of it. But the article seems to suggest it's the WFH transition. How much of the vacancy rate would be explained by that?

For me, the encrypted reasoning contents, when looking at the base64 string lengtht, show this effect. However, the server-reported reasoning tokens don't. So I assumed it was part of the encryption and/or obfuscation purely. So I don't think there is a real issue.

This is the biggest downside of GPT; thinking is encrypted, so it's more of a black box than kimi/glm/deepseek. You still get thinking summaries though. It's awkward, but workable.

There seems to be no awareness from EU govenments about how much power we're handing over to two large outside companies. This incompetence in the leadership will cause a lot of harm over the years. This has been going on for a long time.

That he ran into blockers with the exotic setup is understandable. After overcoming a bunch of challenges, I was surprised to read that the missing Flatpak package stopped him in his tracks, that felt surprising when I read it. It feels like a lesser challenge than the issues he has already addressed.