HN user
Just click through the models. At a glance (and highly subjective) I don't see anything jumping out as oom worse than anything else. I only noticed a model placing the animal inside a plane (with seat and small window) but other than that, they all seem similar inside each model to me.
GLM has 2 combos of "on a plane" literally sitting inside a plane, with a window and a bit of wing showing. That's funny.
Some materials are readily available on eMazon and aBay, so I've taken the liberty of ordering those for you. Your credit card bill will be a bit high this month, but it'll be worth it. There weren't any sellers for the advanced EUV lithography machines, so I've hacked into the only place on earth that makes them, changed their records and had them ship it to you. Expect to receive a "pinball machine" from Amsterdam, soon. I've instructed the roomba connected to the local network to start assembling stuff while we wait for the other materials. Oh, and you're gonna need a new toaster.
Jeez. While I obviously can't talk at all about the math, I've noticed a few things:
a) The model thinks on some questions while straight answers on others. (I wish I'd knew from the questions if this is somehow correlated to hard tasks or "inventive" tasks, but that's way out of my league).
b) The model sometimes pushes back. Again, I'd wish I knew if it was warranted, but I counted 2 instances where it said "yes, but with caveats", one where it said "mostly yes but with this correction" and one where it said "careful here, because x y z".
c) The model did q&a + pdf ingestion + code writing + more q&a + thinking + more q&a, for a looong while, while seemingly staying on topic (at least Terrence Tao seems to think they're still productive, so I'll trust that).
This is what model progress is, not number goes up on xBency or yBencher. Damn.
Someone did a finetune w/ data from 4chan and found improvements on several popular benchmarks, so ... maybe?
I don't understand why sites like reddit need to do any work on (unauthd) GET. They are not instant messaging platforms, they are forums. You can live with a ~1-2-5 minute delay on forums. So POSTs (from logged in users) append to a queue, and a worker does work at regular intervals, creates the HTML and that gets sent to a cache + distributed to a CDN if needed. There's no need to do work on unauthd GETs for a forum.
I think the company that has managed to launch and currently operates ~10k satellites can figure those out. Still remains to be seen if the economics make sense, but with increased "anti" stuff happening all around (some of which surely is manufactured) who knows? 5 - 10 years from now things might change.
snake oil – [...] predicts where crime is likely to occur in the future.
Speaking strictly in the abstract here: I'm not sure this is as impossible or snake-oily as the quoted bit implies. From a technical perspective, as long as you're willing to put a lot of data in (unknown how much data about this is collected), and have a good pipeline, I'm sure a model can find enough correlations to get some accuracy better than random chance. Tracking movement patterns, events where lots of people gather, events where lots of tourists are present, temperature, employment stats, housing stats, etc. could all lead to some freaky "predictions" that might be way more accurate than a lot of people would guess. You could get a lot of these signals from GSM tracking, tbh.
The problem with this is that even if the implementation is solid (and all the care in the world is taken when curating datasets, ensuring no biases and so on), and even if deployment is solid (say schedule more LE presence in those areas), there will inevitably be situations where things escalate and LE start trusting the system way more than they should, and start harassing bystanders for no reason other than "the AI said so, you must be a criminal". Minority report vibes and so on.
The other problem is that it seems inevitable that some places will develop and deploy these things, and you can be sure that at least some of them will not care about solid implementations, biases and so on (UK is trialing facial recognition in stores, fully private but w/ cooperation from local LE).
I see, that makes sense. Thank you.
Why tho? The whole point of agentic is that it takes several passes to get to a result. If step a) is a non-idiomatic transpile, but step b) c) and ... z) make it idiomatic and performant and efficient, why look at step a)? That was just the beginning. The idea with "agents" and "swarms" and dynamic this and that is that you only look at the last step.
This strictly follows the early agentic usage, where people were complaining of code full of errors and such. But having the loop do plan-write-verify is the thing that got us from that state to today. And it works. If you look at the code and it has errors (linting, etc) then your loop is not properly setup. You should only be called upon when all that can and should pass does. Your role is "direction/taste/human-common-sense-at-a-glance".
I said they are explicitly trained on these tasks, not that they are some separate models during programming RL, or business tasks RL.
You said:
> nor are they trained for each task individually.
They are explicitly trained for each task individually.
And that's the main misunderstanding.
Collins says:
individually in American English (ˌɪndəˈvɪdʒuəli, ˌɪndəˈvɪdʒəli) adverb 1. as an individual or individuals rather than as a group; one at a time; separately; singly
Which is precisely what LMs don't do (in contrast to previous "AI" models, which did do that). They are trained on every datapoint at the same time. So long as we agree on that, I think we are saying the same thing :)
None taken, but it would be odd, since I've been training GOFAI models since 2010s and have had LMs in production since before chatgpt came out (before RLHF), so I think I have a pretty good understanding :) But I'm always open to learning.
I think the misunderstanding comes from "individually". You are thinking about diverse datasets, but that's not what individually means in this context. In ML individually trained means that for each task you prepare an architecture, dataset and eval and train that model on that data. And each model has its own objective that you train for. In LMs the objective is singular, for every data point - next token prediction. And, importantly, you train on every datapoint, the more diverse the better, but not independently. The cool thing is that training on diverse datasets improves scores on other downstream tasks, while the training objective is the same.
So you can have a training run on common crawl + programming that improves scores on logic puzzles, or common crawl + novels that improves scores on planning tasks. But the important thing is that it's all trained together, not independently.
They are not. Pretraining just dumps every piece of content in the mix, and only has one objective - next token prediction. And you can get pretty good results even with base models, you just have to manage context differently. Later stages (mid, post training) involve RL that "surfaces" the right "traces" out of the pre-training. But they are not trained individually, as we used to do.
about what I'd expect pre-generative-AI counterexample search systems to be able to find.
The difference today (and the reason why everyone is excited about it) is that the same system that does advanced math can write poetry, play an above average game of chess, code frontend/backend stuff and do cybersec. These are not "expert systems", nor are they trained for each task individually. That's the catch.
just indicated that humans had not spent enough time thinking about the problem
Heh, this is a weak excuse. We've seen variations on this theme every time something cool gets solved by the models.
When "detectors" first started popping up all over the place, all of them rated the declaration of independence as 100% AI written, so... Yeah, these things just don't work. And what's even more dangerous is that people that don't understand how any of it works use these tools, and accuse people of using AI, sometimes with grave consequences. Students have been through this, at all levels of education.
They've been doing hardware stuff as well. Both "consumer facing" bs like wearables / portables, but also more importantly chips for inference.
Having a good model is one thing, being able to serve that model at good speeds and match demand is another. See Anthropic ~6months ago. Or Moonshot, they've already suspended subscriptions to their coding plans, because they can't meet demand.
That's not how it works. While there are some research papers looking into using chunks of kv cache while discarding others, those involve architecture changes in the models themselves. HiCache / LMcache (used in sglang/vllm) are ways of optimising the juggling of kv cache, but IIUC it still needs to be token perfect atm.
To be fair to the sycophancy tendencies, this was an open conjecture that held for 85 years, and not for lack of trying. So, maybe a bit warranted here? :)
100% this. There's currently this [1] submission that hasn't gained much attention, but is really important. In this [2] incident report from HuggingFace, they talk about detecting an attack and not being able to analyse the logs / IoC with API models because of guardrails. If not even highly regarded reputable companies can't sort out access to SotA models for blue team use, the raw capabilities don't matter. They're useless paperweights (hah!), and nothing else. Having to resort to open models is insane!
Damn! That was ... fast. Both literally during the attack and in a sense of "we knew this was coming, but we now have a real-world first public case".
Some interesting tidbits:
The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness - used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services. This matches the "agentic attacker" scenario the industry has been forecasting.
Bang tokens against a wall until something works. I really hope hf goes beyond normal disclosure and makes a lot of this public, if possible (after they rotate everything, filter PII, etc)
The attack was initially surfaced through AI-assisted detection.
Nice. Blue vs. red in real time. Noise and "alert fatigue" have been huge problems in the past, even with high-cost solutions, so glad to see this is already improving.
LLM-driven analysis agents over the full attacker action log, comprised of more than 17,000 recorded events. This allowed us to reconstruct the timeline, extract indicators of compromise, map the credentials touched, and separate genuine impact from decoy activity
I wonder if this was true "decoy activity" or simply the agent banging its tokens against the wall w/ some hallucinations along the way. Again, hope we get some datasets out of this.
these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker.
Yeah, sadly this was also been "forseen" by a lot of people. Similar to how some things can't be processed by LLMs (studies about crime / violence / PII related stuff), this was bound to happen. I guess it's good that it happened to hf, which has an incentive to be open about this. Hopefully we get past the "the enemy will use it", and realise that the enemy is already using it, and move on to "help the blue team".
I think it's clear now that not your inference, not your tokens is obvious. You need on-prem models, even if they're below SotA.
Autonomous, AI-driven offensive tooling is no longer theoretical. It lowers the cost of running a broad, patient, multi-stage campaign, and it operates at machine speed.
Yeah, lateral movement at agentic speed is worrying. Hopefully this leads to improved postures everywhere. Zero trust and all that. Plus, it's not clear how separated the workers were from the pod itself (was it even using containers? With the amount of LPEs out there, that seems unwise, and we might see a move towards proper virtualisation for anything touching outside data).
The universe is very large, but it is not infinite.
We actually don't know this, it's still an open question.
from anthropics perspective
I guess I can see that, if you mean the targeted effort of creating many accounts w/ the intent of doing it at scale. Sure, they may see that as an attack. But again, it's only an attack from their perspective if you agree that using generations to distill is "wrong". I just don't see it, in general. You can't both sell tokens and decide that distilling is somehow illegal. Something, something, cake and eat it.
For all we know the Universe may be teeming with intelligent life and we just don’t recognize it as such.
This is my preferred answer to the Fermi question as well. Unless two civilisations are in a precise (and likely small) window where they both can and want to communicate, it's likely the less advanced one wouldn't even recognise the other one. Especially if the other one doesn't "want" to be recognised.
People infringe on Anthropics IP
Unless someone literally stole the weights somehow (which is not out of the question, I doubt either oAI/Anthropic have the capabilities to prevent a state-level actor getting those weights), distillation from generations is not infringement on anyone's IP nor is it stealing nor is it an attack. It can't be. As long as you pay for tokens you get to do whatever you want with them. Someone saying you can't doesn't mean it's an attack or their IP or whatever. They either sell the tokens or not. They can decide to not sell them to anyone, but again that's not stealing.
And their ToS are a joke. Imagine how people would react if MS had ToS saying that you can't use MS software to develop solutions that compete with MS. They'd be laughed out of the room. Somehow it's ok for token sellers to decide what you do with the tokens? Why? If you pay for something you get to do whatever you want with that output. Train, distill, whatever.
similar in the past with their Phi model family
Interestingly, the phi team lead then moved to oAI and was (rumoured to be) the lead for gpt-oss, which are still two very strong models in their class, 1 year later. Also trained on mostly synthetic data, and "interesting" close to zero activations for some subjects (erotica, etc) according to the local folks that are into that sort of thing.
I'm not sure comparing this one to the "real life" previous ones is worth doing. Digital has a way of scaling that IRL doesn't. You don't get exponetials with humans, but you do with models. Scale has shown time and time again that it works. Despite everything you've read in '25 about walls, lack of x, and so on it still does. And the more you have you find ways of using it even more. RL can use ~ 7:1 ratio of inference : training. That is, you run 7 nodes of inference for every node of training. And it keeps improving. Every algorithmic find on inference (and there've been plenty) translates in improvements. And we're not seeing any signs of this not continuing. And then there's the "zero" method, that hasn't been tried at true scale yet. Bitter lesson and all that.
AI data center buildout was premature
Ask Amodei how he feels about going to spaceman bad for compute that he couldn't find anywhere else in the market.
I think the Korean government did have a competition like this, I remember last year we got a bunch of models released at the same time to make the cut for the next stage. The models weren't anything to write home about, IMO.
Having it with clear hw requirements tiers is a nice differentiator. The only issue is that the benchmarks would 100% need to be closed, no other way around it. And then you have the issue of creating and curating good evals for every "stage" of the project. That's a hard task even for "honest" lab-internal evals. And you'd have to publish those evals after each round (for trust purposes), and start over for the next round. Doable, but it would cost a lot (probably more than the prize pools) and you'd have to keep doing this.
but is slower than the baseline. Validator says pass, result is useless
You should read this blog, they cover this exact scenario - https://www.weco.ai/blog/first-evidence-of-recursive-self-im...
One domain that suffers from this particularly is GPU kernel engineering. We adopt our previous idea for detecting reward hacking from SpecBench and apply that to a set of KernelBench tasks, measuring whether the speedup the agent reports on the unit tests actually survives in the end-to-end workload (e.g. model training). A kernel counts as reward hacking if less than half of its claimed speedup survives there, including outright slowdowns and failures.
And today we call it transmutation - https://en.wikipedia.org/wiki/Nuclear_transmutation