galsapir

thanks for reading it properly and engaging with the argument!

writing is hard, expressing ideas cleanly is harder! working on it.

curious where the disagreement lands: the claim i'm least sure of myself is that measurement alone already counts as activation (nothing in the weights changes, so it's a looser sense of the word than usual) the part i'd defend harder is the eval -> reward one: once a benchmark becomes the thing you train against, its flaws stop being measurement error and start being incentives. if you're pushing back somewhere in there, i'd genuinely like to hear it

really interesting that its basically almost 80% claude opus..

yeah its really counterintuitive i think; i.e, getting the right framework and structure for this to work probably isn't trivial, models really hate playing well together. i wonder how their version would fair in real world use.

i feel like i've had exactly the same thought in the past :-0 might even have written about it. feel your pain

sometimes I also feel it tries to optimise for "per line coverage" over more "real, complex use cases" type tests

hey that's pretty cool. I think I still prefer "distill HN" cleanliness though. What made you create this.

axon discharge is brilliant. adopting.

oh sorry! didn't catch the one Thanks, I'll comment there

From the link: "Shot from 90 perspectives, 88 focus stacked images each. Nikon Z8, full frame, f/7.1, exposure 1/160, ISO 100, Laowa 180mm macro lens, with LED light and bluescreen." Insane!

I think the question he tried to raise was "is this needed? Aren't today's / tomorrow's models well-enough equipped to deal with just OPEN API?" (idk, just if I understand the question)

got me at "Most often scientists believe they understand more than they do, making their belief an illusion." but why is it still bothering me? 1. feels unfalsifiable in spirit 2. somewhat restates "all models are wrong, but some are useful" less cleanly 3. doesn't really offer like, what can we do as science people? tomorrow morning perspective

author here. the part i'd actually like discussion on is the buried finding: physicians+GPT-4 didn't outperform GPT-4 alone on the management cases, and on the landmark cases the model alone beat the model+physician. the paper reports it and moves on. that's the 2026 question, and it's the one a Science-level platform could have been used to ask

Hey thanks! I do wonder that. I think that even if specifically for code smell the things would be subtler, for other forms of AI driven averageness (especially in areas where we can't RLVR the models to perfection) it might still be present. But yeah I wonder how those thoughts will age (and how we'll update our priors accordingly).

yeah I was really thinking about what the best "umbrella term" would be here. Since "LLM" is too widely used in a really specific context and "AI systems" felt niche I ended up with "LMs". Idk, up for debate..

haha that's a style choice (takes more work to get lowercase text these days). But yeah legit ;-)

Thanks! I'll check it out.

one of the most interesting pieces I've read recently. Not sure I agree with all the statements there (e.g. without execution the system has no comprehension) - but extremely cool

The part that made me laugh out loud: Dostoyevsky's description of medicine becoming "too specialized" — one doctor for the right nostril and another for the left. That's from a conversation between Ivan and the devil. Written in 1880. The rest of the novel is like that too — the narrator is semi-omniscient but explicitly unreliable and self-conscious about it, the characters' inner lives contradict their stated beliefs, and the psychological model overall is more sophisticated than most of what we use in social science today.

nice, esp. liked - "our memories, our thoughts, our designs should outlive the software we used to create them"

this seems interesting, do you have an example of a use case that you found it helped with? (Red green pattern where without RUNE, it failed)?

I've been writing about how I use AI tools as an researcher working in health AI — specifically the tension between leveraging them and staying engaged enough to catch when they're wrong. This post is about a specific version of that problem: the models have gotten good enough that my default is to trust the output, and the threshold for "worth checking" keeps drifting upward. So I built a simple Claude Code skill that sends high-stakes work to a different model family for a second opinion — one call, not a multi-agent debate. The honest result: the first real test (reviewing an architecture spec) scored maybe 6/10. It caught one genuine security finding and missed the deeper domain questions entirely. That gap maps onto something I keep running into in evals — tools can check structural form (missing error handling, security anti-patterns) but struggle with essence (does this actually work the way the spec assumes? are the clinical guardrails robust?). Still worth it as a lightweight intervention against the drift toward not checking at all. The skill is open source if anyone wants to try or improve it.

haha nice for freelance work!

opus 4.6 came out yesterday so I tried it and built two things. i think the model is smoother: picks up intent faster, better questions in interview-style flows, more willing to loop for 8+ minutes. the tools: an interview command for claude code with depth checkpoints, and a markdown annotator for actually reviewing what comes back instead of staying in the "fix it plz" loop.

There's been a lot of discussion around this lately, especially after the Anthropic study. I don't have answers — this is more an attempt to articulate the problem and some mental frameworks that have been useful. Curious what practices others have found helpful

we spent a few months building evals for a health agent (and the agent itself!). tried to apply anthropic's framework to a real system looking at CGM data + diet. some of it worked. we got decent at checking form — citations exist, tools were called, numbers trace back. the harder part was essence — is this clinically appropriate? actually helpful? we didn't really solve that. curious if others building health/bio agents have found ways around this, or if everyone's just accepting fuzzy metrics for the stuff that matters.

foundation models in biology still haven't proven they're worth it vs simpler methods (imo). we just published one in Nature, and i feel i spent more time on "how will we know this worked" than on the model itself. the hard part was (mostly) deciding what success even means. open for thoughts

i honestly dont think there's a simple y/n answer there - i think considerations include mostly like 'how costly it is to do so', 'how often do you think you'll need it', and so on. traces are not as "ephemeral" as FT models - since you can use those to guide agent behaviour when a newer model is released (but still, not as evergreen as other assets - traces generated using say GPT4 would seem pale and outdated compared to ones created on the same dataset using Opus4.5 i reckon)