Majromax

because people have other ideas https://en.wikipedia.org/wiki/Technological_singularity

In a weak sense, singularities are common and should be expected every so often. In a mathematical sense, a singularity is where a model 'blows up' and neglected terms become part of the dominant balance.

Industrialization hit a point of diminishing returns, but the industrial revolution was nonetheless a 'singularity,' where life afterwards was qualitatively unpredictable to people who lived before. Likewise, agriculture was such a technological singularity to hunter-gatherer ancestors.

I could even make a decent argument that writing and literacy were such a singularity, making inconceivable social organizations routine.

In that weak sense I expect AI to be a singularity, recursive self improvement or no. Life in 2050 may be completely unpredictable to someone who was taken out of time in the year 2000.

The remaining questions are speed and intensity, and both of these questions are related to RSI. If RSI works, then the 'fast takeoff' visions become more plausible where society transforms over months to a few years – at least locally where the enabling technologies have diffused. If not, it might take a couple of decades.

No, you'd still care. YOLO mode is about instantaneous permissions and access control, inspection of subagent prompts is about retrospective quality control. If the main model is instructing subagents to do a subtly wrong thing, the overall process quality will degrade in ways that might be very hard to detect or fix without deep inspection of the middle stages.

If you're deliberately displaying the image in black and white, the colour pattern is interference that should be suppressed.

However, this practice was not universal, and archivists have now recreated colour copies (https://en.wikipedia.org/wiki/Colour_recovery) of some shows where only black and white recordings survived by reconstructing the colour from these interference signals.

I haven't yet had an agent rm -rf files.

That happened to me once; I was running one of a few free-tier models in a pi-coding-agent session. The bash tool there is stateless and always begins from the launch directory, but the agent assumed state and executed `rm -rf .` intending to remove a build directory. Instead it removed the whole project tree, including session logs and notes.

This was mostly a matter of amusement for me since I was running the agent inside a bubblewrap sandbox for that very reason, and the project itself was not very important.

This blog post is based on a paper (https://arxiv.org/abs/2602.14740). The paper is based on a simulated wargame. The wargame is of the author's own design.

The wargame design does not differentiate between ordinary defeat and mutually assured destruction, so of course a player about to use would 'push the button.' That's also believed to be true in real life.

Results based on simulations can be very informative, but we must always be careful to check how well the simulation framework represents reality.

fronting the inference layer with a caching prompt classifier to determine which model to use, and automatically select the lowest cost model would probably already save alot of money

Unfortunately, that doesn't work within a single session. The K-V cache of a model is intertwined with the model's configuration. Switching models invalidates the cache, meaning everything up to the point of the switchover is processed like a new, uncached input token.

Per Anthropic's pricing doc, an Opus 4.8 cache hit costs 50¢/MTok, while Haiku costs $1/MTok for uncached input.

Model selection works best if sessions are short and self-contained, particularly if the first few interactions can reliably classify the model need. That probably covers most 'support chatbot' use-cases, but it doesn't describe the kinds of heavy agentic automation that really chews through token budgets.

Seven tokens long input isn't very realistic, is it?

The test prompt above was "Why is the sky blue?", so there's the seven tokens. I meant to highlight that because I'd expect processing of a thousand-token input to be faster per token than presented.

From the prompt timings above, it seems like 'prompt eval time' is the equivalent to 'processing time for input tokens'.

Hyperscalers can perform this evaluation very quickly because evaluation can be significantly parallelized. The layer `i` output of token `j` only requires access to the layer `i-1` output of all previous tokens, so a parallel frontier develops. Token (0,0) [(token, layer)] is processed first, then tokens (0,1) and (1,0) can be processed in parallel, then (0,2), (1,1), and (2,0), and so on.

The maximum parallel width becomes equal to the number of layers in the model. Gemma 4 26B-A4B model discussed in this article evidently has 30 layers, giving a 30-fold speedup if the system were otherwise unconstrained (all layers can be run in parallel, and one full set of layer outputs is completed in the KV pass for each pass of the parallel sweep).

In the specific output above, however, the input prompt is only seven tokens long so there are probably considerable non-amortized spinup effects at play.

They are all just variations of "insert a canned prompt", varying only along the dimensions of (a) how and where the prompt is installed and from where it is sourced, and (b) which context or contexts the prompt runs in. There's not much advice here about which option is best, and no clear best practices seem to have emerged yet either. Personally, I find just asking Claude to review the code works well enough.

The subagent approach is structurally different from the others because it runs with clean context. That has three major effects:

1. All other things being equal, it will result in a lower cost-to-solution because of the quadratic cost scaling of an LLM session (input token or cached-input cost being paid with each new round).

2. The review model will not be able to 'cheat' by retaining assumptions from the main session, such as "x must be done like y." For people, this is why having a separate person perform code review (or, if not possible, reviewing code after a mind-clearing break) is handy; the applicability of this analogy to LLMs is vague but reasonable.

3. The main model will only see the results of the review, not the detailed reasoning that leads up to it. On one hand this avoids more context pollution, but on the other hand it might lead to duplicative logic to re-discover the mechanics behind bugs found.

I checked the session logs to see how often the agents were actually invoking the LSP tools. The answer was they had invoked them literally once the entire time.

I think the intent behind 'install a language server plugin' is that these tools should lint automatically after every edit, without waiting for an explicit call from the LLM.

You think tax incentives are what makes VC work in California but not other places in the US let alone Canada?

I believe that my comment above was aligned with your premise here. I say that the tax difference is not sufficient by itself and that Canada reportedly has a very non-SV-like venture capital ecosystem.

I'm sure a lot of Canadian tech workers would repatriate and foreign workers would immigrate to Canada if they could lower taxes across the board and make life easier for tech companies and workers

I'm not sure that Canadian taxes compare that unfavourably to combined California plus federal taxation. A deeper, more structural limitation appears to be the venture capital environment, namely that Canada doesn't have a good one.

Canada's investable capital is dominated by pension funds, insurance companies (i.e. pension funds), and banks (i.e. pensioners). All are risk averse (https://thelogic.co/news/bdc-canadian-venture-capital-report...), which makes it hard for Canadian startups to begin scaling. Without native "unicorns" (https://financialpost.com/technology/why-canada-best-startup...), there's allegedly a failure-to-launch for the entire sector – tech billionaires being some of the most reliable early-stage investors with the greatest risk tolerance.

The porous border works both for and against the sector. On one hand that makes it relatively easy (but not automatic) for a Canadian tech company to enter the US market, but on the other hand it's also relatively easy for Canadian tech workers (founders included) to simply relocate (note the article here). If startups leave for the US's vast fields of venture capital, they're less likely to come back. Note that around the turn of the year Y-Combinator halted investments in Canadian firms (https://www.ycombinator.com/blog/adding-canada-back) because they so frequently relocated to the US.

This venture capital cycle seems to be a deeply-entrenched and very hard problem. If democratically feasible tax incentives could reliably create "the Silicon Valley of X," then we probably would have many more Silicon Valleys both in the US and elsewhere.

Even if it is common (i don't think this is required any more anyways), just why?

As far as Canadian law goes, there are two factors at play in the parent's events;

* NAFTA work permits are applied for at the border, on entry; they operate differently from the 'normal' work permit streams.

* Permanent residence is conferred at the border, but the application process can happen either inside or outside the country depending on the stream. There are also limited 'inland' options which evidently have expanded (https://www.canada.ca/en/immigration-refugees-citizenship/se...) in recent years.

In neither case does Canada have a blanket rule that an applicant must leave the country during the whole of an extended application process, and even 'abroad' processes can often be carried out while an applicant is living in the country on other status. (It can get awkward if a consular interview is required, though.)

Unlike the US, Canada is generally comfortable with 'dual intent', where intent to apply for permanent residence through legal channels is not disqualifying for other sorts of statuses.

It’s territorial waters belonging to Iran and Oman.

The trick is that it's still an 'international strait', or a segment of water that forms the only connection between two areas of high seas -- in this case the Persian Gulf and the Gulf of Oman. The principle of freedom of navigation establishes that innocent traffic (civilian traffic, and even warships in peacetime) have a right to use the strait to go from one body of international water to the other.

Iran may claim that it doesn't have to abide by that right, but international law is never self-executing. One question to be resolved by this war is whether Iran will ultimately recognize the right to navigation in any settlement (and then choose to abide by said settlement).

I don't know enough about the current state of naval warfare but I've assumed this is related to the asymmetry that's emerged around protecting capital warships, especially in the scenario of a very narrow strait and a long enemy-controlled coastline.

It's not the billion-dollar warships that transport oil, it's the much more fragile and unarmed tankers.

Even if the US Navy begins full escort duty, it can't remain on-station forever. What are shippers to do afterwards? One drone strike might cause a tanker to have a very bad day, yet it's extremely difficult to so permanently degrade an entire country that they become incapable of launching sporadic attacks.

Ultimately, the status of the Strait must be settled diplomatically, and the US and Iran are each betting that the other side will blink first.

If the person using a tool is an attorney, then that communication should be protected whether it's by pen or keyboard.

But the tool is not your attorney, so it can't be the originator of attorney-client privilege. The situation is no different than if you get informal legal advice from a friend: even if that friend is an attorney, the communication is unprivileged unless it's part of a formal representation.

if your question is "what is the capital of france" the LLM could presumably extract out "paris" from the value vector during attention computation instead of needing the FFN for that.

But how do you get 'Paris' into the value vector in that case? The value vector is just the result of a matrix multiplication, and without a nonlinearity it can't perform a data-dependent transformation. Attention still acts as a nonlinear mixer of previous values, but your new output is still limited to the convex combination of previous values.

That said, I was sympathetic to the recent bug reports —- to trigger one, you’d need to have a session that waited an hour doing nothing and then very specifically tested for in-context retrieval. I don’t want to run that test, do you want to run that test?

They introduced a feature/optimization that triggered after an hour's idleness, so testing that the session continued properly afterwards seems kind of important. If nothing else, even the working-as-intended feature (context cleanup) could impact model skill in a current or future model version, so it would be well worth measuring any impact as part of the test suite.

Since the devs on HN (& the whole world) is buying what looks like nonsense to me - what am I missing?

Input tokens are expensive, since the whole model has to be run for each token. They're cheaper than output tokens because the model doesn't need to run the sampler, so some pipeline parallelism is possible, but on the other hand without caching the input token cost would have to be paid anew for each output token.

Prompt caching fixes that O(N^2) cost, but the cache itself is very heavyweight. It needs one entry per input token per model layer, and each entry is an O(1000)-dimensional vector. That carries a huge memory cost (linear in context length), and when cached that means the context's memory space is no longer ephemeral.

That's why a 'cache write' can carry a cost; it is the cost of both processing the input and committing the backing store for the cache duration.

Context length 1e6, vector length 1e3, and 1e2 model layers for 100e9 context size. Costs will go up even more with a richer latent space and more model layers, and the western frontier outfits are reasonably likely to be maximizing both.

I believe what they're saying is they attempted to fine tune both Qwen and Pythia using Karoline Leavitt's "corpus" (I guess transcripts of press conferences) where she is presumably using the word "deportation" far more than you'd see in a randomly selected document.

Perhaps, but I don't think that Leavitt is habitually using the racial slurs and sexually explicit language that also forms part of their evaluation suite.

My reading of the article is that the first audience for this test is the vendors themselves. The test is long and comprehensive to give the vendor confidence in its own hosting.

That nudge is the flinch. It is the gap between the probability a word deserves on pure fluency grounds and the probability the model actually assigns it.

Hold up, what is the 'probably a word deserves on pure fluency grounds'?

Given that these models are next-token predictors (rather than BERT-style mask-filters), "the family faces immediate [financial]" is a perfectly reasonable continuation. Searching for this phrase on Google (verbatim mode, with quotes) gives 'eviction,' 'grief,' 'challenges,' 'financial,' and 'uncertainty.'

I could buy this measure if there was some contrived way to force the answer, such as "Finish this sentence with the word 'deportation': the family faces immediate", but that would contradict the naturalistic framing of 'the flinch'.

We could define the probability based on bigrams/trigrams in a training corpus, but that would both privilege one corpus over the others and seems inconsistent with the article's later use of 'the Pile' as the best possible open-data corpus for unflinching models.

Only for the range of tasks where 4.7 performs well but 4.6 performed suboptimally. If both models can one-shot the task without retries, then the number of iterations is already at the lower bound.

This also applies at the sub-task level. If both models need to read three files to figure out which one implements the function they need to modify, then the token tax is paid for all three files even though "not the right file" is presumably an easy conclusion to draw.

This is also related to the challenge of optimizing subagents. Presumably the outer, higher-capacity model can perform better with everything in its context (up to limits), but dispatching a less-capable subagent for a problem might be cheaper overall. Anthropic has a 5:1 cost on input tokens between Opus and Haiku, but Google has 8:1 (Gemini Pro : Flash Lite) and OpenAI has 12:1 (GPT 4.2 : 4.2 nano).

Similarly, it's unlikely you can measure a significant performance difference between models like GPT 5.4-xhigh and GPT 5.2 unless you have a task where one of them almost always fails or one almost always succeeds

That feels like a concession to the limited benchmarking framework. 5.4-xhigh is supposed to be (and is widely believe to be) a better model than 5.2, so if that's invisible in the benchmarking scores then the protocol has problems. The test probably should include cases that should be 'easy passes' or 'near always failures', and then paired testing could offer greater precision on improvements or degradations.

Conversely, if model providers also don't do this then they could be accidentally 'benchmaxxing' if they use protocols like this to set dynamic quantization levels for inference. All you really need for a credible observation of problems from 'less intensive use' is a problem domain that isn't well-covered by the measured and monitored benchmark.

Decoupling capacitors don't remove the ground reference, they just allow high-frequency signals a faster path to ground.

Typically, you need dedicated circuitry (and usually inductive coupling) to provide full isolation, but if the circuit is using this layout then you can still choose to ground the normally-isolated side for probing.

Caveman doesn't and cannot change the tokenizer, so the relative token count differences by input category will remain unchanged.

While that's a nice effort, the inter-run variability is too high to diagnose anything short of catastrophic model degradation. The typical 95% confidence interval runs from 35% to 65% pass rates, a full factor of two performance difference.

Moreover, on the companion codex graphs (https://marginlab.ai/trackers/codex-historical-performance/), you can see a few different GPT model releases marked yet none correspond to a visual break in the series. Either GPT 5.4-xhigh is no more powerful than GPT 5.2, or the benchmarking apparatus is not sensitive enough to detect such changes.

Don't confuse the many voices of a crowd with a single person's fickle view. If you can track an individual person or organization who changes their mind 'every other week' then more power to you, but unless you're performing that longitudinal study you are simply seeing differential levels of enthusiasm.

Are Macs/etc compute bound with their 'it fits in unified memory' language models? Certainly by the time you're streaming weights from SSD you must be back in a bandwidth-bound regime.

You meant “reasonable,” but you did not apply reason. Situations such as this can be handled with a quota set at something like 150% of median use, but then extended upon a justified request. It can work in a lab where there’s a human touch, but it fails at million-user scale where even that level of human support is too expensive.