brunooliv

Genuine question: can these posts be paid to hype the open source models? If yes, what would be the purpose?

On my work tasks, FastAPI Python and Springboot Java on a modern SaaS product, the only open model that can do tasks well and efficiently is Qwen3.7-Max.

In all my experiments, both GLM-5.2 and Kimi are busy grepping around the codebase for ALMOST 70-80K tokens before writing anything and when they do it typically breaks the code… it feels to me that these models are good but only when you write out a super detailed spec of the task just like it was done a year ago… Qwen3.7 just… does it

Qwen is so much better than GLM or Kimi that this makes me genuinely excited!!

I hope you're right and I want you to be right, but, even seeing the current hype around local models, etc... and open-source models, I think the industry is currently under a big confusion where they see the benchmarks of things like Kimi, GLM, Qwen, they play with it via opencode, and they think like: "Wow this is pretty good, I want to deploy this". But they don't understand how the KV cache grows over time and can take almost as much memory as needed for a 30B param model, they dont understand that a quantized model WILL NOT be the same as a full precision one, and they surely don't see the engineering work needed to serve inference to even tens of customers at a decent quality and latency level.

The biggest moat of these giant labs and models is increasingly shifting towards deployment capabilities and (debatably) having better (proprietary) harnesses.

The models themselves can be impressive on benchmarks, but unless they can be served reliably to customers either at scale, hosted somewhere, or even on edge with predictable latency and memory usage, then frontier will always be leading.

This is really insane to me.

There's nothing practical about open-source models yet that makes them even remotely comparable to closed frontier models.

All the hype around GLM, Qwen, now Kimi.... Are people really this naive that they believe these reports or, more worringly, are people NOT using these models and seeing the HUGE gap that still exists?

Take a task, any medium-sized task, decently scoped that you'd trust to give to Sonnet to finish without a hitch. Now give it to ANY open-source frontier model and watch them struggle and go in circles while failing tool calls and randomly assuming things.

Open-source is and has been amazing but its so hard to deploy reliably and at scale and there's still big problems in the underlying models with instruction following and tool calling that makes it basically unusable for production workloads at a decent price point...

So on one hand it states:

Delivery is becoming a sequence of checks, not a ritual The same thing happens when it is time to deliver.

First, I invoke my /definition-of-done skill. It checks whether the implementation covers what was stated in the PR and in the plan. It checks tests and the other validations I care about. It tells me when something looks unusual, broken, or missing. When everything looks good, I invoke another skill, /pr-check-release. That checks the remote PR, updates labels, removes [WIP], adds [RFC], updates the description, and prepares the change for review. If one day passes and nobody on the team reviews the PR, we merge it. The agent also tracks that condition for me, so I can run /pr-merge-dev, and it takes care of the process: merge the PR, delete the remote branch, delete the local branch, and pull dev back into a fresh state. None of these steps is particularly difficult. That is exactly the point.

But then:

I still care about all of those things. Probably more than most people.

Obviously the author has irreversibly became AI-pilled and the day API costs balloon or APIs are down, what work will the author do?

I love using AI but please read the diffs and process them with your human brains and eyes. Spin up your containers manually, test the app, MANUALLY. Talk to real users face to face.

Outsourcing the grunt work is fine, but there's a fine line between that and becoming a button-presser.

I only wish Opus 4.6 from earlier this year at a faster inference speed. Since Opus 4.6 things have been so much messier and the overall push for more agency isn’t really panning out for agent assisted development as much as they would like

Thanks for saying this! I completely agree with everything you said!

There’s far, far too many people who confuse code quality for speed of development and start treating code quality as the product for customer base in the hundreds and active customers in the dozens and for most features to be basically unused.

The reality is that tech debt as a concept these days is hardly real: to be in debt means previous decisions or a previous implementation makes current work extremely hard or impossible, but, the truth is that the human factors such as knowing what to build, team collaboration and even speaking to customers matter far more and can get you “in debt” so so much faster than code alone. At least in your typical SaaS company.

If you ship code in a way that you let tech debt pile up to the point that customers notice it, you have an organisational problem, not code issues per se.

The fact that a lot of people don’t get this is really baffling to me.

I’ve been plugging away at my running coach style app, powered by the original idea of training for a trail race while living in a flat area with no easy access to natural climbing that has evolved into a fully functional plan generator: https://runcoach.fly.dev

It works well for me so far and I’m pretty happy with it!

Side question: I've always been a recreational runner, running 3/4x a week, completed a few half marathons, and recently decided to _also_ go to the gym to do strength training as it has a lot of benefits for runners too. Should I consider/take creatine, is it useful?

What if you then use AI to try and maintain only one, a single product into which you’ll put your care and craft to try to make something that’s better than “some dopamine hits”?

Having tried many of these hosting services to host/play with toy apps, DigitalOcean and Fly.io are both unparalleled GOATs.

Any reason why they indexed on Kimi K2.5 model? I have tried many open-source ones in Opencode, and, in my experience (standard backend development, Java, Python, Spring, etc) Qwen3.6 is SO MUCH BETTER that's shocking. Kimi can't even get most tool calling arguments right.

I still haven’t seen any other models be as complete as Claude inside Claude Code. I bet Anthropic knows this and they turn the knobs and see people’s reactions… I have been planning with Qwen3.6 Max inside opencode, absolutely game changer. Opus can then follow the plan quite detailed and like this I can make progress on my toy apps on Pro plan at 20/mo.

For work, unlimited usage via Bedrock.

Yes I’d like to get more usage out of my personal sub, but at 20/mo no complains

Also my experience

Maybe… but I can say I saw a real shift in these last few days, why or if it’s real, I can’t fully say but definitely something changed

I’ve been using Opus 4.6 extensively inside Claude Code via AWS Bedrock with max effort for a few months now (since release). I’ve found a good “personal harness” and way of working with it in such a way that I can easily complete self contained tasks in my Java codebase with ease.

Now idk if it’s just me or anything else changed, but, in the last 4/5 days, the quality of the output of Opus 4.6 with max effort has been ON ANOTHER LEVEL. ABSOLUTELY AMAZING! It seems to reason deeper, verifies the work with tests more often, and I even think that it compacted the conversations more effectively and often. Somehow even the quality of the English “text” in the output felt definitely superior. More crisp, using diagrams and analogies to explain things in a way that it completely blew me away. I can’t explain it but this was absolutely real for me.

I’d say that I can measure it quite accurately because I’ve kept my harness and scope of tasks and way of prompting exactly the same, so something TRULY shifted.

I wish I could get some empirical evidence of this from others or a confirmation from Boris…. But ISTG these last few days felt absolutely incredible.

In the end, Anthropic is a company and needs to make money, my best bet is that even those of us who pay 100/mo to use Claude Code are costing Anthropic money, besides all the rest they’re burning on inference.

Again, I agree with you and the service should be at least reliable but to be completely fair, if I had to bet, the amount of usage people get for 100/mo is probably only balanced out by the corporate/entreprise customers paying their bill to Anthropic via API usage.

If we look at it through this lens, this limits are not surprising at all, except maybe on how generous they are/were. It’s pretty obvious that they want to force people to pay as they go….

Ah yes this is sad to see and a lame move for sure… It’s indeed dependent on usage hours but it’s a bad move even if I’m personally not affected since I use it outside of those hours but I agree it’s lame….

Is that documented somewhere? Do you have a link? I might have just missed it, and if they did it, I will take my words back.

I think in this case, we probably have different experiences that shape how we see some things differently: I see many (very smart) people doing certain things that are not optimal (eg: copy-paste entire files instead of referencing them or tell claude at every message to "read CLAUDE.md and follow its instructions precisely") which can lead to a lot of token waste. If certain system prompts were tweaked internally or some models now read more files than before, keeping these "inneficient prompts" will make limits exhaust faster. Sub-agents or this new agent teams feature didn't exist until a few months ago: that alone eats A LOT of tokens, not intended for this pre-paid API usage, etc.

The ecosystem is evolving super quickly so, our own experiences and workflows must keep adapting with it to experiment, find limitations and arrive at the "tightest possible scope" that still allows you to get things done, because it is possible.

Another example: pre-paid monthly subscription aggregates usage towards web and Claude Code, for eg. So if you're checking for holiday itineraries over your lunch break, then decide to sit down and ask a team of agents to refactor a giant codebase with hundreds or thousands of files, context will be exhuasted quickly, etc, etc.

I see this "context economy" as a new way of managing your "mental models": every token counts, and every token must bear its weight for the task at hand, otherwise, I'm "wasting budget". I am also still learning how to operate in this new way of doing things, and, while there have been genuine issues with Claude Code, not every single issue that people encounter is an upstream problem.

It's the nature of SaaS software, right? It doesn't need to be an enforced "hard change", but, let's say that they trained Opus 4.6 to be more "verbose" or to explore more files to gain more context for it's own tasks.

If your limits stay "the same", but you then use Opus 4.6, your quota will be exhausted much faster, it's just how it works.

Note that some features are simply NOT made for these Pro, Max, Max 5x or whatever pre-paid plans. I'm pretty sure this is by design and not an accident or a bug: If you have 6/7 MCP servers configured or if you want to use this new feature of "Agent Teams", you will exhaust your entire quota before ANY work is even done. This is not a bug. Each agent has its own context window and tools and they all count separately.

MCP servers, when active, add A LOT of context to your sessions before you even use them, etc, etc.

It feels to me that people want to have their cake and eat it too, but, that would NOT be a sustainable business model. You can not complain about the tools if you can't understand them in-depth.

I want to state that I don't think Anthropic are fully aware of the ramifications that ANY small change in ANY of their models might have, because their entire ecosystem is a bit messy atm, but, I'm certain they're aware that if people dont like it, they will cancel the subscription and flock to a competitor very quickly, since there's no real moat anymore. So, it's in their own interest to keep things minimally usable even on the "cheaper plans".

I have seen people with 5-10 "active MCP servers" that they "wanted to try out" then they forget about it and wonder why their context is always full... Cmon... that's almost bad faith.

I don't fully defend Anthropic as they've had several issues with degraded model quality after releasing "the latest model", and CLI usability that cost me real money and real tokens, so, there's a lot of room for improvement, but, to claim that quota gets exhausted after 1h it points out to either some forgotten MCP servers, skills or giant files being accidentally read in, or some sort of mis-use which these limits were put in place to prevent exactly.

There's a very thin line between: quota is exhuasted on a regular, normal session after 1h and I think there's a bug versus I had 3-4 MCP servers active that I am not using at all but forgot to disable and my CLAUDE.md file is 1000 lines...

Unusable if not Opus 4.6 on max effort sadly. Price is quite steep too! I still remember when Sonnet was an absolute beast…

Replying once again for future reference to make my position clear: I firmly believe that one MUST experience programming on its own first. No LLMs, no crutches. One MUST feel the abstractions melting away and things clicking in the brain first.

The design becoming obvious. Being able to remove that extra if statement after clarifying requirements with a customer face to face.

A design pattern fitting a scenario like a glove, etc, etc.

You need REAL experience that only comes with time and effort. Years or decades, different businesses, different companies, etc.

But once you have crossed that chasm and that rite of passage, using LLMs becomes a true multiplier and my experience quite fun.

Using them blindly or without experience is a very different thing I can imagine.

I like problem solving and building useful things for our customers. Coding for me was always more of a “means to an end” than pure craft on its own. Obviously some standard, good and clean code pops up when you’re working in things to be extended or maintained by others, but, truth be told, ego battling in code reviews gets boring very fast and additionally, no matter how much I like experimenting with things, if I have an hypothesis, I can now validate it in 2 days instead of 1 week, which means I can validate double the hypothesis.

I am extremely excited about that! Coding in itself as the act of manually typing things? Absolutely not

Couldn’t read the entire comments but, my experience has been overwhelmingly positive so far. I think what helps me be effective is a combination of factors: I work only in a modern, well-documented and well-architected Java codebase with over 80% test coverage.

I only use Claude Code with Opus 4.6 on High Effort.

I always, ALWAYS treat my “new job” as writing a detailed ticket for whatever it is I need to do.

I give the model access to a DB replica of my prod DB that I create manually.

I do NOT waste time with custom agents, Claude.md files or any of that stuff.

When I put ALL of the above together, the results ARE THE PROMISED LAND: I simply haven’t written a single line of code manually in the last 3 months.

What rock?

C'mon let's be real here, there's either "testing AI skills" versus "using AI agents like you would on the daily".

The signal got from leetcode is already dubious to assert profeciency and it's mostly used as a filter for "Are you willing to cram useless knowledge and write code under pressure to get the job?" just like system design is. You won't be doing any system design for "scale" anywhere in any big tech because you have architects for that nor do you need to "know" anything, it's mostly gatekeeping but the truth is, LLMs democratized both leetcode and system design anyway. Anyone with the right prompting skills can now get to an output that's good for 99% of the cases and the other 1% are reserved for architecs/staff engineers to "design" for you.

The crux of the matter is, companies do not want to shift how they approach interviews for the new era because we have collectively believed that the current process is good enough as-is. Again, I'd argue this is questionable given how sometimes these services break with every new product launch or "under load" (where YO SYSTEM DESIGN SKILLZ AT).

I really love Anthropic's models, but, every single product/feature I've used other than the Claude Code CLI has been terrible... The CLI just "sticked" for me and I've never needed (or arguably looked in depth) any other features. This for my professional dayjob.

For personal use, where I have a Pro subscription and adventure into exploring all the other features/products they have... I mean, the experience outside of Claude Code and the terminal has been... bad.

Would be really curious to see

Mickens is the best!

It should be fixed now!