HN user
Jagged frontier is not the same as being benchmaxxed. Benchmaxxed is à la Goodhart's Law "when a measure becomes a target, it ceases to be a good measure." Jagged frontier is about how models that seem superhumanly intelligent at one category of tasks (e.g. coding web applications" can seem toddler level or worse at another category (spatial reasoning) because the training corpus doesn't generalize to there.
Haha, this is awesome. Hadn't heard of it, thank you for the link.
If you haven’t been really running and testing these models yourself, they are all benchmaxxed. No matter how close they score to frontier on whatever metric, they always fall apart in real world tasks and their token efficiency is ridiculously bad. Fireworks has incredible incentive to make this claim in a headline, because Fireworks hosting K3 for you is pure profit for them, unlike when they host closed source models.
Very high quality and info-dense article. Cheers.
This is really smart, like the author said, old idea but cleverly applied.
In case anyone wants a summary: don’t one shot, don’t use plan mode and hand off the plan to cheap executors, ask the frontier model to explore, create a todo list, and then start when it feels confident; stop it after first code edit, then prefill the context to cheap executor to continue.
Everyone using Claude Fable to verify this proof is so funny. If you read the definition of the Jacobian Conjecture and (I am not exaggerating this) have passed a college Calc 3 class, you can just verify the proof yourself in 30 seconds. The problem was very hard to solve but the counterexample is very easy to verify!
--- edit, adding an explanation:
To summarize it, the conjecture says if you have any multi-variable polynomial function that maps an input to an output in the same dimensional space (take for example: F = (x+2, y+2), which maps 2D space into another 2D space), AND that function has a constant-valued non-zero Jacobian determinant, THEN the conjecture is that the polynomial has an inverse, meaning basically you can find a polynomial that turns the output space back into the input space.
Fable provided the example polynomial (which was very hard to do) and the coordinates which if you plug into it, results in two points being mapped to the same output point. This means that the polynomial can't be inverted, because if you have that output point, how do you know which input point it came from?
You can just plug in the two coordinates it gave into the equation and verify that you get the same output point from both. That's the contradiction of the conjecture and it takes 30 seconds.
---
Something something outsourcing of thinking something.
Eh, Minimax M2.7 also took a similar amount of time (actually longer) between availability and weights release.
you could live (100k a year) with the interest of 0.004% of that in the bank.
Apple (and the ARM ecosystem as a whole) has never really needed massive GPU compute before, it’s always been about power efficiency and just enough GPU oomph to make UI fluid. Even historic Mac Pro workloads never really needed tons of GPU prowess, the heaviest power users were primarily taxing video encode/decode and 2D raster effects (Adobe Suite etc) so that’s what they focused on. By contrast NVIDIA’s entire game has been raw GPU power for decades. M1 Max was really prescient in hindsight and has set the stage for, all things considered, Apple to be not as far behind as it could have been today.
I think we should make it illegal to not specify the quantization in the headline for these types of posts.
this is really cool but it seems very unlikely that someone targeting an exotic system not supported by rust (mostly embedded and ancient mainframe targets) would be willing to trust a beta transpiler to not inject any bugs or leaks in the process of turning rust to c. nevertheless, very cool.
An important idea I’ve observed true across industries is “prices rise like a rocket and fall like a feather,” meaning that even though price rises are usually genuinely driven, you can bet that once they’re up the involved parties are doing whatever they can to keep them up. Humans are greedy.
More formal reading: https://en.wikipedia.org/wiki/Asymmetric_price_transmission
this is really clever, props
Location: NYC / LA
Remote: yes
Willing to relocate: NYC or LA
Technologies: Motion planning, autonomy, embedded microcontrollers, embedded Linux, perception systems, controls. C, C++, Python, Rust
Résumé/CV: https://github.com/nabeelsherazi/nabeelsherazi.github.io/blo...
Email: see resume
7 YOE in shipping robots to customers, lots of safety critical stuff, looking to move into urban autonomy (self driving cars, delivery bots, construction bots, humanoids, etc). Looking for a startup with a mission I can get behind.
they’re doing m7 on the intel 18a fab, which is exactly that
One of the stupidest things about this is we talk all day along about how frontier models don’t just interpolate distribution, then can extrapolate out. Then something like this comes along and a model can generate gore or CSAM so therefore there must be gore or CSAM in the training data. Eye roll.
My honest read is that, having everything — the data centers, the compute, the models (however misaligned they might be), the only thing xAI is missing is users. They don’t have any users because the only people who use Grok are essentially Elon’s fanboy club, and all they pretty much do with it is ask it to generate arguments to win their Twitter threads or nonconsensually unclothe people. Cursor gives xAI a captive audience of users; most sophisticated users don’t use it anymore, so anyone left is unlikely to be opinionated when models are shifted to Grok. Marriage made in heaven.
this being HN, from the title i genuinely had no idea whether this link would be about music, the apple graphics acceleration framework, or ore deposits.
really impressive. did not expect this from infineon.
One of the lessons of philosophy is that once you adopt any particular value system, almost all philosophers either become immoral or caught up in meaningless and trivial quibbles.
Can you explain more about this?
Do people have extremely complex Actions that I can't fathom?
Yes. Think CI jobs that test every candidate PR against a matrix of build targets, run fuzzing, run simulation tests, run bench regression tests, etc etc. Modern CI workflow automation has reached way beyond what a pedestrian can fathom if you’re not on the wave.
I had issues with Qwen thinking endlessly when I didn’t know I wasn’t using the temp/top_k/min_p/etc settings specified in the readme. I’ve never had an issue with Gemma 4 thinking endlessly but could possibly be the same.
This makes a lot of my experience with Qwen make sense. I’ve watched all the benchmarks imply how close it should be to various GPT or Claude releases, but in my own use chatting with it or trying to get it do agentic tasks it was nowhere near as smart as even GPT-3.5 for example. Meanwhile Gemma 4 casually dropped and even the 4B models were performing better than Qwen 3.5 MOE in my chats. Benchmaxxing.
Perennially relevant: https://www.lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimu...
Also in comic form: https://stuartmcmillen.com/comic/supernormal-stimuli/
not anything new. people have been taking up “old timey” hobbies to get away from screens since the late 2000s. you’ll remember the hipster javascript baristas of portland taking up leatherworking and blacksmithing between launching meteorJS sites “made with <3”
In his entire screed he never realizes the reason Apple got their transition to work was they genuinely worked hard to make it happen on both ends: they forced developers hard AND still shipped Rosetta 2 to make it seamless for user anxiety for laggard developers. They even had this playbook watching Apple do their first transition from PowerPC to x86. Yet he seems to think Windows 8’s problem was velocity.
it’s not ZIRP anymore but it might as well be. you can truly get funded to make anything right now
I believe you’re late to the “ash shell” name by about 36 years
very interesting article but i was surprised pg’s conclusion was the opposite of what i expected. to me it was like oh this is brilliant, instead of trading effort linearly for money you can just control image and be paid outside for it. reminded me of, e.g. chamberlain canned coffee, which tastes terrible but has emma chamberlain’s personal brand lifting for it in the aisle every time you see it.
you never know what’s going on in someone else’s claude max plan