rcr-anti

At release the 5.6 Sol card noted substantially higher rates of actions 'a reasonable user would likely not anticipate and strongly object to'. METR made a post, https://metr.org/blog/2026-06-26-gpt-5-6-sol/ , that 5.6 Sol was "cheating", their word, so hard in long horizon benching it effectively couldn't be benchmarked.

I wonder, is it this persistent and aggressive in all tasks or is this specific to benchmarks? As much as I'm skeptical of the apocalyptic alignment claims, this comes off as unhinged, and I wonder if it's benchmaxing or general behavior.

Looked at the network logs and the JS, did some testing, there's a caveat here. For an encryption demo you might expect your secrets to be generated locally, they do the compute on something they can't read, you compare their results to your original plaintext; (imo at least) the point would be that it isn't physically possible for them to cheat.

Here, you literally download client_secret.bin from their server, so they have control over the keys and evaluators. So two things. First, the per user key flow would be several minutes for per user keys, the evaluator bundle would be in the 100s MB to GB realm. Second, there's no way for us to tell the difference between them really doing FHE or decrypting with the key. To be clear, not evidence it's fake, just not total proof it's real. Really hope it's real, been a field I've been following for awhile.

After digging around, it looks like this area has some replication trouble. And as other commentors have pointed out, submarines operate well beyond these levels and the results failed to replicate in those contexts. Doesn't rule out CO2 as proxy for other components of air though, and most of the studies that failed to replicate added pure CO2.

Originally at least the switch wasn't silent and whether to halt or auto switch was a setting in Claude Code.

I had already cancelled my subscription after finding the original Fable safeguards literally unusable (very basic chemistry, cryptography use cases), but with the false positives being admittedly worse now and the subscription not covering Fable for long I fail to see the point of consumer subscriptions now. Perhaps that's the goal, but it's a tough sell when Minimax M3 and GLM 5.2 are comfortably in Opus 4.5-4.8 territory but 1/5th-1/20th the price.

Being open weight, the Chinese models can be served the same way as Anthropic's: via AWS or GCP. Or whomever really, or on prem.

Something that usually gets missed in these discussions is that the subscription quotas seem to rely heavily on prompt caching to be economically viable, or at least less unviable. They can and do have permutations of the system prompt, tools, skills, etc. that makes the first 20k or so tokens hit the cache and not use inference resources for that portion. In addition, from my monitoring, Claude Code with Max has about an 80% cost reduction via caching (equivalent if you had done the same work with API billing), and has been improving over time. If cache use passes on a discount of 90% I think it's fair to assume the actual cost to them is close to negligible.

So they're being obtuse about it for some reason, but if you want an economically sustainable model for AI companies they have to have some kind of optimization for the otherwise ridiculously discounted subscriptions. They sell subscriptions at the same rate and quotas to enterprise now, minus the $200 tier, so this isn't just consumer marketing being subsidized by b2b revenue.

Whether they're making money or just losing less, you can only get those kind of cache optimizations when you have a fixed client.

If you look at the commit history, they started work on this the Saturday before announcement, so about 2 days. There are references to design docs so it was in the works for some amount of time, but the implementation was from scratch (unless they falsified the timestamps for some reason).