HN user

p1necone

10,433 karma
Posts2
Comments2,718
View on HN

I've noticed GPT specifically has more of a tendency to stop partway through things than many other models do. Although my most recent experience with it was 4.X I believe.

Hearing "here's what I've done, here's the completely unambiguous next steps, I'll wait for you to send a pointless message before I continue" over and over again is a real pain.

Qwen 3.8 3 days ago

I use GLM-5.2 as an orchestrator model which delegates to Deepseek subagents (v4 flash or pro depending on complexity) and it works pretty well for a quite complex compiler codebase when I do deep enough up front planning for features/fixes.

If you believe the benchmarks Deepseek v4 is pretty shitty at long running work in large codebases, but really really good at self contained algorithmic/math reasoning - which is basically ideal for a compiler for a language with a relatively complex type system. And with its cache pricing it's very cheap.

GLM-5.2 is too expensive at api pricing for my taste though even when it's not producing the bulk of the output tokens - I pay for the mid-tier subscription and switch the orchestrator over to other models via open router when I run out - Minimax M3 feels okay but definitely a step down from GLM-5.2.

This worked for Snow Crash - felt like I went on a theme park ride, got to the end of it, and hopped off still buzzing. Doesn't work so well for something less tongue in cheek.

I don't know about Kimi, but in my experience other providers like z.ai and minimax don't serve the same models via their subscription plans as they do with api pricing.

They're clearly quantised - spelling mistakes, wonky thinking section dividers, missing whitespace are the immediately obvious tells but along with that comes degraded quality, and it seems to vary based on time of day.

I wouldn't be surprised if Kimi did similar things with their subscription plans - try using it through openrouter and see if you notice different behaviour.

"The kids are alright" is evergreen. So much of the <insert current generation who are teenagers> are doomed rhetoric is pure nonsense, a lot of that rhetoric is on here.

there's something funny about mathematics in that every novel result is broadly perceived as a big deal.

Is this true? Or is it just that mathematics is an isolated enough field that only the results that are a big deal get broadcast widely to the public.

I know little of the inner workings of the field of mathematics, but my naive assumption would be that there's probably lots of novel but boring results being discovered/proven all the time and we don't hear about them because no-one outside of the person doing the work and a handful of their colleagues is really that interested in it. Likely a lot aren't published in any way, because they're just stepping stones towards the goal of the actual area/paper/whatever being worked on.

Everything else feels bloated. The same model in pi will consume significantly less context for the same task as there's not piles of unchangeable system prompts being passed along (I assume), and at least compared to OpenCode the quality feels better too. I just wish Anthropic let me use my Claude subscription with pi, it's annoying having to switch between them.

(Not to mention the unmatched level of customisability)

If you've got money to burn on tokens, the way that seems best to me is to set up a repeatable harness - docker container with a specific past commit from your own project, set of known issues/features that you've already fixed/completed of varying levels of complexity.

Set up a script that launches the harness for each model, prompts them to implement one of the tasks, let it churn until either tests pass or it hits some budget limit.

Then, most importantly, read the transcript and output and judge subjectively - I don't think this actually can be narrowed down to a score, although tokens burned to fix, whether it actually got the tests green etc are all good signals.

(I've done this, but so far only on a codebase that was too complicated with models that were too weak because I didn't want to spend more than a few dollars - results were inconclusive, planning on iterating on my personal benchmark in future)

The stuff about scale not allowing quality resonated with me.

baseless speculation follows!!!

I think large orgs can definitely achieve high quality. but only by spinning up small, totally autonomous teams working on every layer of whatever stack their product is on, one team per product (maybe two if there's some really obvious line in the sand between two different things that talk to each other, but be careful! and make sure both teams are in the same timezone!).

As soon as you start trying to do those things that seem really sensible when you have a bunch of separate autonomous teams - like "hey you're both working on similar features, you should share the implementation", and "oops all our products look different, we should come up with a unified component library", and "we need automated tests - everyone should use this specific tool that we paid for" you run into the big org problem.

My gut feel is that the best way to get some level of coherency without running into these problems is to share knowledge, best practices, examples etc. But never dictate anything that actually gets in the way of any of your teams owning their own shit. Don't make teams use some internal/external library for functionality x, don't enforce processes, don't have a separate design team dictating css styling to teams, don't enforce org wide CI policies, don't have a separate DevOps team handling releases - just hire competent people and let them do their thing. If you do want to try to build something so that all the teams solve the same problem in the same way, you need to get them to use it by making it so good they want to, not by telling them they have to.

You might be able to enforce some baseline level of mediocrity by doing those things, but the only way to achieve excellence is to get out of the way and stop trying to "help".

My father was a watchmaker. Fond memories of going with him in his van to the various jewellers he did work for picking up and dropping off. I remember being given a big metal lamp from his workshop when he passed away and realising the body of the lamp was not isolated from the incoming power, although luckily not at mains voltage (not what killed him).

Anecdotally, the people who I know who were not particularly good developers pre-llms still manage to produce bad code even using flagship models now.

I think having solid knowledge/understanding of good architecture and general practices is still crucial, and it's easy to forget that the foundational knowledge and instinct you take for granted now actually took a lot of time and effort to learn when you were less experienced.

Eh I think you're overstating the level of lock-in here. There's nothing about the platform itself that forces you to stay, it's just the games you've already bought. You can keep the system and just stop buying new games on it.

And most people probably don't replay most games after beating them once, for the handful you do want to replay, you just bust out that console occasionally, or you grab them again in a steam sale to play them on PC instead or something.

Ever since I had the disposable income to burn (so ~PS3 generation onwards) I've been someone that always eventually owns every console from each generation. Usually just one of them initially and then the others a few years later when they get cheap.

I think I'm going to stop buying new game consoles and games now and just stick to PC (mostly via Steam) and emulation. Might continue with switch stuff if they keep publishing proper physical media. I find myself revisiting console games that came out 20+ years ago often, and that's a large part of the reason I buy them.

It's still putting my trust in an online store, but I have much more confidence that my Steam games will still be usable a couple of decades from now than the xbox/ps stores. And historically Nintendo has totally axed the online store for each console when the new one comes out, including killing the ability to redownload already purchased games, although they did retain the same store between Switch 1 and 2.

Will be interesting to see whether Steam stays safe when Gabe eventually dies/retires. I think if he passes it on to a trusted successor it might be good for another few decades+, but if the company gets sold or publicly listed I have no hope.

(As a side note, if anyone is a console gamer primarily because of the nice "turn it on, get controller, play with no stuffing around" vibe of it - a pc in a small case with bazzite-deck installed that's just permanently hooked up to your TV is a very seamless experience, it's remotivated me to play a lot of stuff on PC that I would normally default to buying on consoles - JRPGs, racers, platformers etc.)

I completely lost interest in Magic when they started churning out all the licensed crossovers. It feels like they really jumped the shark compared to how they used to be - they used to seemingly care deeply about the lore, gameplay, structure of releases etc whereas now it all just feels like a cash grab.

Usually I agree with your calls on things being unsubstantive, but this one kinda seems fine? I don't think it's flame bait, just emotive language? And the substantive point being made is that online gambling should be illegal.

(apologies if arguing about mod decisions is frowned upon, I didn't see anything in the rules about it)

Are you seriously trying to claim that e.g. wearing dresses or liking the color pink is somehow fundamentally tied the the genitals in your pants or the chromosomes you have?

The idea that gender is a social concept is so blindingly obvious that, like bbeonx I kind of assume that anyone making comments like yours about "common sense" is either blindly parroting talking points without thinking about them, or arguing in bad faith.

Basically saying that women and men should present and behave within a narrow set of parameters.

I think you're putting words into peoples mouths there.

Acknowledging that there is a social construct we generally know of as "gender" and acknowledging that certain stereotypes and common understandings of that concept exist is not at all the same thing as demanding that people should fit into the narrowest stereotypes that you can think of.

Also worth noting that you acknowledging the existence of sexist stereotyping is an acknowledgement of the existence of gender as a social construct.

I think you need to read their comment again - they are clearly talking about sex and gender as two different concepts.

Yeah my current feeling is that once I had double digit millions earning further money would be pretty meaningless to me, and the difference between 'large salary' and 'even larger salary' would be even more meaningless, but who knows maybe it really would change me. I kind of assume people like this are primarily chasing the most interesting/impactful work though.