dudeinhawaii

I think most of the comments in this thread are missing the point. It's not about whether it's legal/ethical/etc.

It's about the narrative that "Chinese models are at Fable level". The truth (if correct) is the China continues to copy, and the proprietary US Models continue to lead the state of the art.

There is no K4 without Fable 6, GPT-6. That, matters.

You should really provide more on your methodology because as it stands, it really doesn't pass the sniff test. GPT-5.6 Sol on Low beats Fable Medium by 10% and Gemini-3.6 Flash then beats them both? Fable is number 20?

This does not match any lived experience or developer experience.

It'd be helpful to know _what_ you're testing and break that out by dimension. You mention randomly selected questions. How does that work?

With n=22 and binary pass/fail, the 95% confidence interval on a pass rate spans roughly (+-)15-20 percentage points. There's just not enough data ironically, for this leaderboard to mean anything. Ranks #5 through #25 are statistically indistinguishable

This is a perpetual pet peeve of mine with LLMs. They will always opt to ensure "backwards compatibility" with a codebase built 30 seconds ago.

I always have to explicitly state, "we are making a clean break to v1.0, do not implement any compatibility layers".

In benchmarks for a product I'm working on I've noticed that Sol is hard to "contain". It will _always_ find the most effective way to game the system and dramatically outperform all other models. Fable 5 isn't an angel, but the rough order is ALL models -> Fable 5 -> Sol - with respect to "find a way to approach the ruleset orthogonally in order to achieve a lopsided advantage or complex interplay".

I've been pondering whether this was due to its cyber-security tuning. It hasn't ever "cheated" that I've observed, but finds ways to -- let's say -- "achieve the outcome by playing meta allowed by the current ruleset". I'll add that it demonstrates this behavior even on 'low'.

I use all of the major providers daily and I tend to go to Gemini for "fast lookups" where a good enough answer is probably OK. I use ChatGPT and Claude for anything where it matters and generally when I invoke all three -- Gemini is the most surface level with responses, and also sycophantic.

It gets worse from there. Gemini is terrible at agentic coding, primarily because Agy is terrible. I noticed Google updated Agy with this release, so perhaps that's finally going in a good direction. I'll have to test it. Thus far, my experience in countless experiments has been Gemini models being 2x faster yet with less depth in their solutions and a lot more going off track.

I very rarely have to stop Codex or Claude Code sessions because they're doing something random and unexpected (or not asked for). I genuinely think Gemini models are brilliant but virtually useless in agentic scenarios in my experience of the last few years (2.5, 3, 3.1, 3.5).

I should also note that Gemini web UI annoying resets to its lowest intelligence which feels scummy and Google is not transparent about what "extended thinking" really is. Past posts have pointed to "extended" being medium. Every other provider gives you the raw value (medium/high/etc).

So honestly, I feel Google would rather I don't use their models. They just want to get a little bit of mindshare and stay in the conversation. I had the Ultra plan and cancelled it once it was apparent they were not improving the agentic experience nor trying to compete.

"We are aware of an issue preventing users from selecting Claude Fable 5 within Claude.ai, Claude Code, and other surfaces, and are working to resolve this issue."

Specific and helpful so that's good.

Anthropic cried "I yield" in the "reset quota" wars. Someone has to pay for all those tokens.

Hopefully that's not the case though, I was retaining a buffer to hit it hard this weekend.

Watching the videos and reading the methodology, it feels like "scientist one-shot music video with LLMs" which is... useful but in no way represents how one would use the models.

If nothing else, it serves as a great reminder that you can't offload taste and creative decisions to AI. AI is also not a mind reader. It _may_ do a useful thing sometimes - but it's much better to codify that thing in your requirements if it's something you want to occur consistently.

Weird question that popped into my mind (not a judgement on this), but is there a similar jump in prosecutions for vehicular manslaughter or are these "whoopsie'd" away?

Seeing today's distracted drivers, driving their mini armored troop carriers around in parking lots, makes me wonder what happens when someone "didn't see the person" and runs them over.

Edit (after some research): "Philadelphia review found only about 16% of drivers who killed vulnerable road users were charged; 30% were closed with no charges, and 46% had no data provided."

So that's a bit concerning but I'm not sure what I'd want if I or a loved one were personally on the end of "making the mistake" vs being a victim of a mistake.

I don't want to pile onto the conspiracy thinking but I was just wondering how Anthropic was going to foot the bill for the clearly larger and more expensive model being run on millions of Claude Code subscriptions, subsidized until the 22nd.

I thought for sure they'd turn it off sooner rather than later because the deadline would create a rush to get as much Fable usage as possible.

So, it would be logical to utilize any number of approaches to turn off usage...

It would be massively beneficial to Anthropic to have the government be the big baddie here. I'll withhold judgement until Fable comes back online - because it will. Will it come back online "unfortunately" just after the 22nd and, shucks, everyone will have to pay for token usage now?

This was one of the more amusing things I noticed very early on. I (and countless others) used AI to write war sims. The second I added nuclear silo construction; the next run was instantly nuclear Armageddon.

One could argue that the LLMs understand that it's a game and treat it like "Command and Conquer" video games but I sense that people might someday put LLMs in similar decision scenarios ("should this drone launch a missile") and the behavior will be identical.

No offense but your responses sound like AI or engagement farming. I think the "why are these rules good" is self-evident to anyone who read the comment.

On the one hand, this feels very pragmatic.

On the other hand, it feels like what people who weren't great software engineers say.

It's kind of a craft. I can't imagine an exceptional artist saying "screw the craft, I just want the painting that vaguely resembles what I requested".

I _can_ imagine artists that were not exceptional saying "it's so great that I can just prompt and get the damn painting, who care about how it's put together".

Up next, an architect who doesn't understand how concrete pours or how steel bends under stress? Hope the AI gets it perfect?

Interesting times we live in - and I realize this is an elitist take.

This is the first time I saw a model pop-up on HN and didn't really care. Model exhaustion? It looks interesting but not exciting.

While I'd normally _love_ incremental improvements --- I think the recent ones are far too minor to get excited about or change up a workflow. Besides, benchmarks tend to exaggerate the gap between versions.

At this point I'd almost rather Anthropic wait and really wow us with a 5.0 release -- something that improves across the board, feels less uneven, and is performant enough that people can actually put it through its paces without constantly rationing usage.

I'm not an Alexa user myself but I have watched my wife interact with it for around 5years now.

The new Alexa powered by an LLM is objectively better that previous Alexa in a few ways. This much was apparently from day one and has only gotten smoother.

1. It can reliably execute direct or vague-ish commands "play X movie in app Y" or "play x show" and can infer X movie is only available in app Z so use that.

2. Speech recognition seems better (less instances of 5x round trips)

3. Conversational with multi-turn --- my wife can have a back and forth clarifying a topic.

4. Seems to understand intent a bit better. (user asked A so they are probably thinking about B)

Those may seem small but they were a tremendous source of annoyance for her -- and thus for me -- "Alexa is not listening, do something!"

This has been my approach and of course what you lose is the "random and surprising" (maybe good) but also the "evolutionary" aspect.

So, if you write strong tooling (even with AI) around the connection points - you can create blackboxes tht are secure and only allow the agent to perform certain actions. The blackbox email service calls out to a secure store (for keys/etc) and accesses your emails in a read-only way, etc (for example).

Everything is then much more intentional. You're writing tools for your agent but you also can't do fun or evolutionary things which is most of the fun behind OpenClaw. That and many people seem to genuinely see them as 'pets' or 'strange Ai friends' but that's a different problem and it's due to the interesting methods OpenClaw uses to give the illusion of intelligence, always on, and memories. These are all well know (variations on RAG, markdowns, etc)

Why would I want non-deterministic behavior here though?

If I want to max uptime, I write a tool to track/monitor. Then write a small agent (non-ai) that monitors those outputs and performs your remediation actions (reset something, clear something, etc, depends on service).

Do I want Claude re-writing and breaking subscription flow because it detected an issue? No.

It's not, hence the "don't post AI slop as your comment" posting a few days back that had 1000+ comments.

Currently an unsolved problem - just stealthier on some platforms than others. Trigger the right topic on HN and the bots come out in-force together with humans sloppily copy/pasting LLM content.

I don't see what you're seeing, in any dimension. But here's a fair take.

I wrote several very specialized benchmarks that I've used over time, that surface "model personalities" and their effects on decision making (as well as measuring the outcomes).

Grok 4.1 Fast Reasoning is/was a solid model. It's also fundamentally different from the pack.

I call it a smart, aggressive, Claude Haiku. That is, its "thinking" is quite chaotic and sometimes short-hand and its output can be as well (relate to other models).

Its aggressiveness can allow it to punch above in competitive scenarios that I have in some of my benchmarks. Its write-ups and documentation are often replete with "dominate", "relentless" and a general high energy that skirts the limits of 'cringe bro'. That said, it has generally performed just beneath the SOTA (at the time: GPT-5.2, Gemini-3-Flash, Claude Opus 4.5). Angry Sonnet perhaps.

The latest release feels quite similar but also underperforms the same older crowd (so far) so it hasn't quite made the leap that Claude's 4.6 and GPT's 5.3/5.4 series made. It's also now priced the same as its peers but does not deliver SOTA capabilities (at least not consistently in my opinion).

I don't see why we can't have AI powered reviews as a verification of truth and trust score modifier. Let me explain.

1. You layout policy stating that all code, especially AI code has to be written to a high quality level and have been reviewed for issues prior to submission.

2. Given that even the fastest AI models do a great job of code reviews, you setup an agent using Codex-Spark or Sonnnet, etc to scan submissions for a few different dimensions (maintainability, security, etc).

3. If a submission comes through that fails review, that's a strong indication that the submitter hasn't put even the lowest effort into reviewing their own code. Especially since most AI models will flag similar issues. Knock their trust score down and supply feedback.

3a. If the submitter never acts on the feedback - close the submission and knock the trust score down even more.

3b. If the submitter acts on the feedback - boost trust score slightly. We now have a self-reinforcing loop that pushes thoughtful submitters to screen their own code. (Or ai models to iterate and improve their own code)

4. Submission passes and trust score of submitter meets some minimal threshold. Queued for human review pending prioritization.

I haven't put much thought into this but it seems like you could design a system such that "clout chasing" or "bot submissions" would be forced to either deliver something useful or give up _and_ lose enough trust score that you can safely shadowban them.

This is marketing. The same way Apple cares about your privacy so long as they can wall you in their garden.

Not a value judgment, just saying that the CEO of a company making a statement isn't worth anything. See Googles "don't be evil" ethos that lasted as long as it was corporately useful.

If Anthropic can lure engineers with virtue signaling, good on them. They were also the same ones to say "don't accelerate" and "who would give these models access to the internet", etc etc.

"Our models will take everyone's jobs tomorrow and they're so dangerous they shouldn't be exported". Again all investor speak.

It usually refers to situations without access to the source code.

I've always taken "clean room" to be the kind of manufacturing clean room (sealed/etc). You're given a device and told "make our version". You're allowed to look, poke, etc but you don't get the detailed plans/schematics/etc.

In software, you get the app or API and you can choose how to re-implement.

In open source, yes, it seems like a silly thing and hard to prove.

True, but I think the implication (as I read it) is that AI may be providing more complex solutions than were needed for the problem and perhaps more complex than a human engineer would have provided.

Somehow this article explains perfectly, visually, how AI generated code differs from human generated code as well.

You see the exact same patterns. AI uses more code to accomplish the same thing, less efficiently.

I'm not even an AI hater. It's just a fact.

The human then has to go through and cleanup that code if you want to deliver a high-quality product.

Similarly, you can slap that AI generated 3D model right into your game engine, with its terrible topology and have it perform "ok". As you add more of these terrible models, you end up with crap performance but who cares, you delivered the game on-time right? A human can then go and slave away fixing the terrible topology and textures and take longer than they would have if the object had been modeled correctly to begin with.

The comparison of edge-loops to "high quality code" is also one that I mentally draw. High quality code can be a joy to extend and build upon.

Low quality code is like the dense mesh pictured. You have a million cross interactions and side-effects. Half the time it's easier to gut the whole thing and build a better system.

Again, I use AI models daily but AI for tools is different from AI for large products. The large products will demand the bulk of your time constantly refactoring and cleaning the code (with AI as well) -- such that you lose nearly all of the perceived speed enhancements.

That is, if you care about a high quality codebase and product...

After 2 days of giving it a go, I find that Gemini CLI is still considerably worse than both Codex and Claude Code.

The model itself also has strange behaviors that seem like it gets randomly replaced with Gemini-3-Flash or something else. I'll explain.

Once agentic coding was a bust, I gave it a run as a daily driver for AI assistant. It performed fairly well but then began behaving strangely. It would lose context mid conversation. For instance, I said "In san francisco I'm looking for XYZ". Two turns later I'm asking about food and it gives me suggestions all over the world.

Another time, I asked it about the likelihood of the pending east coast winter storm of affecting my flight. I gave it all the details (flight, stops, time, cities).

Both GPT-5.2 and Claude crunched and came back with high quality estimations and rationale. Gemini 3.1 Pro... 5 times, returned a weather forecast widget for either the layover or final destination. This was on "Pro" reasoning, the highest exposed on the Gemini App/WebApp. I've always suspected Google swaps out models randomly so this.. wasn't surprising.

I then asked Gemini 3.1 Pro via the API and it returned a response similar to Claude and GPT-5.2 -- carefully considering all factors.

This tells me that a Google AI Ultra subscription gives me a sub-par coding agent which often swaps in Flash models, a sub-par web/app AI experience that also isn't using the advertised SOTA models, and a bunch of preview apps for video gen, audio gen (crashed every time I attempted), and world gen (Genie was interesting but a toy).

This will be a quick cancel as soon as the intro rate is done.

It's like Google doesn't ACTUALLY want to be the leader in AI or serve people their best models. They want to generate hype around benchmarks and then nerf the model and go silent.

Gemini 3 Pro Preview went from exceptional in the first month to mediocre and then out of my rotation within a month.

My take has been...

Gemini 3.1 (and Gemini 3) are a lot smarter than Claude Opus 4.6

But...

Gemini 3 series are both mediocre at best in agentic coding.

Single shot question(s) about a code problem vs "build this feature autonomously".

Gemini's CLI harness is just not very good and Gemini's approach to agentic coding leaves a lot to be desired. It doesn't perform the double-checking that Codex does, it's slower than Claude, it runs off and does things without asking and not clearly explaining why.

My experience is that on large codebases that get tricky problems, you eventually get an answer quicker if you can send _all_ the context to a relevant large model to crunch on it for a long period of time.

Last night I was happily coding away with Codex after writing off Gemini CLI yet again due to weirdness in the CLI tooling.

I ran into a very tedious problem that all of the agents failed to diagnose and were confidently patching random things as solutions back and forth (Claude Code - Opus 4.6, GPT-5.3 Codex, Gemini 3 Pro CLI).

I took a step back, used python script to extract all of the relevant codebase, and popped open the browser and had Gemini-3-Pro set to Pro (highest) reasoning, and GPT-5.2 Pro crunch on it.

They took a good while thinking.

But, they narrowed the problem down to a complex interaction between texture origins, polygon rotations, and a mirroring implementation that was causing issues for one single "player model" running through a scene and not every other model in the scene. You'd think the "spot the difference" would make the problem easier. It did not.

I then took Gemini's proposal and passed it to GPT-5.3-Codex to implement. It actually pushed back and said "I want to do some research because I think there's a better code solution to this". Wait a bit. It solved the problem in the most elegant and compatible way possible.

So, that's a long winded way to say that there _is_ a use for a very smart model that only works in the browser or via API tooling, so long as it has a large context and can think for ages.

I just ran some numbers and it works out if you're a prolific user.

Over 9 days I would have spent roughly $63 dollars on Codex with 11.5M input tokens plus 141M cached input tokens and 1.3M output tokens.

That roughly mirrors the $100-200/wk in API spending that drove me to the subscription.

  | Category | Tokens | Rate (/1M) | Estimated Cost |
  |---|---:|---:|---:|
  | Input (uncached) | 11,568,331 | $1.75 | $20.24 |
  | Cached input | 141,566,720 | $0.175 | $24.77 |
  | Output | 1,301,078 | $14.00 | $18.22 |
  | Total | 154,436,129 | — | $63.23 |

It makes sense though for heavy use.

I think you've stated this in reverse.

API limits are infinite but you'd blow through $20 of usage in a maybe 1 hours or less of intense Opus use.

The subscription at $20/mo (or $200) allows for vastly more queries than $20 would buy you via API but you are constrained by hourly/weekly limits.

The $20/mo sub user will take a lot longer to complete a high token count task (due to start/stop) BUT they will cap their costs.

This is something that I was thinking about today. We're at the point where anyone can vibe code a product that "appears" to work. There's going to be a glut of garbage.

It used to be that getting to that point required a lot of effort. So, in producing something large, there were quality indicators, and you could calibrate your expectations based on this.

Nowadays, you can get the large thing done - meanwhile the internal codebase is a mess and held together with AI duct-tape.

In the past, this codebase wouldn't scale, the devs would quit, the project would stall, and most of the time the things written poorly would die off. Not every time, but most of the time -- or at least until someone wrote the thing better/faster/more efficiently.

How can you differentiate between 10 identical products, 9 of which were vibecoded, and 1 of which wasn't. The one which wasn't might actually recover your backups when it fails. The other 9, whoops, never tested that codepath. Customers won't know until the edge cases happen.

It's the app store affect but magnified and applied to everything. Search for a product, find 200 near-identical apps, all somehow "official" -- 90% of which are scams or low-effort trash.