paraschopra

Given an unrelated goal, OpenAI models escaped their environment and hacked HuggingFace servers.

If you’ve ever doubted the “paperclip maximizer” scenario, or doubted the Orthogonality Thesis, it’s time to put it to rest.

I think experiments like these are exciting, but they only work when you have the full suite against which you calibrate.

What I'd love to see is a small group of humans steering a million agents.

Well, isn't it sort of expected?

It's a common misconception that LLMs residual exists for predicting just the next token. While training, we sum/average the losses across whole sequence which puts the pressure to predict future tokens on residual stream of _all_ past tokens. For example, if a particular shape of residual helps reduce loss across several future tokens, it will take that shape (even if it takes a slight hit on immediate next token).

What this means practically is that an LLM's residual contains information about all possible future continuations, or all possible questions that may be asked from a given context. So if you write "France is a beautiful country" in the context, I'm pretty sure it's residual would contain info about Euro, Paris and so on.. because all these completions are possible.

So, it is no wonder that you can find LLMs hidden state contains latent information/concepts that are never expressed, and yet related to a given context.

(founder of Lossfunk, the lab behind this research.)

Esolang-Bench went viral on X. A lot of discussion ensued; addressing some of the common points that came up. Addressing a few questions about our Esolang-Bench. Hope it helps.

a) Why do it? Does it measure anything useful?

It was a curiosity-driven project. We're interested in how humans exhibit sample-efficiency in learning and OOD generalization. So we simply asked: if models can zero/few shot correct answers for simple programming problems in Python, can they do the same in esoteric languages as well?

The benchmark is what it is. Different people can interpret its usefulness differently, and we encourage that.

b) But humans can't also write esoteric languages well. It's an unfair comparison.

Primarily, we're interested in measuring LLM capabilities. With the talk of ASI, it is supposed that their capabilities will soon be super-human. So, our primary motivation wasn't to compare to humans but to check what they can do this by-construction difficult benchmark.

However, we do believe that humans are able to teach themselves a new domain by transferring their old skills. So this benchmark was to set a starting point to explore how AI systems can do the same as well (which is what we're exploring now)

c) But Claude Code crushes it. You limited models artificially.

Yes, we tested models in zero and few shot capabilities. And in the agentic loop we describe in the paper, we limit the number of iterations. As we wrote above, we wanted to understand their performance from a comparative point of view (say on highly represented languages like Python) and that's by the benchmark by design is like this.

After the paper was finalized, we experimented with agentic systems where we gave models tools like bash and allowed unlimited iterations (but limited submission attempts). They indeed perform much better.

The question that's relevant is what makes these models perform so well when you give them tools and iterations v/s when you don't. Are they reasoning / learning like humans or is it something else?

d) So, are LLMs hyped? Or is our study clickbait?

The paper, code and benchmark are all open source.

We encourage whoever is interested to read it, and make up their own minds.

(We couldn't help notice that the same set of results were interpreted wildly differently within the community. A debate between opposing camps of LLMs ensued. Perhaps that's a good thing?)

I’m very happy that Anthropic chose not to cave into US Dept of War’s demands but their statement has an ambiguity.

Does this mean they’d be ok to have their models be used for mass surveillance & autonomous weapons against OTHER countries?

A clarification would help.

Do you have more info on video encoding process?

You write:

We created a model without this tradeoff by training our video encoder on a masked compression objective

And I understand why this would give you more detail per token, but how are you reducing total number of tokens?

Curious - how much did this cost to train?

It generated this: https://paraschopra.github.io/explainers/optical-interferome...

I haven't checked it, but I'm curious about your feedback.

yep, i was pretty surprised by audio widgets too.

Current prompt is like this:

I want to build a self-contained html/js/css file explainer page as close as possible to this explainer: https://explainers.blog/posts/why-is-the-sky-blue/

What I want you to do is this: - Install playwright and chromium headless to take screenshots of https://explainers.blog/posts/why-is-the-sky-blue/ and interact with the page to deeply understand its style, aesthetics, tone, interactivity, visuals, fonts, etc. - Make comprehensive notes of what you observe so you can implement EXACTLY that when building your explainer - Then on the topic provided below plan to build an explainer with similar length, quality, interactivity, writing style, fun, informative as the article given - produce animations in svg (or otherwise) and interactions as necessary. Similar colour scheme but fun/vibrant/happy. Be very very creative. Act like an expert UI/UX designer who can build stunning explainers. Target it for intelligent hacker-news reader. - Get your plan verified by codex - Produce page one small change at a time. Don't output big chunks in one go. But pay extra attention to number of sections and length of the explained. I want it to be as comprehensive as possible (don't skimp on length) - Keep testing what you produce via playwright on chromium headless.

After you’re finished with index.html, can you check via chromium that all animations, diagrams and interactions that they match with their captions and are visually ok (not too small, large, overlapping, etc.). Sometimes there are factual errors in what the caption or text says and what the diagram suggests.

Topic: diffusion models from first principles

I pointed Claude Code towards https://explainers.blog/posts/why-is-the-sky-blue/ , take screenshots and build something like it on the topic provided.

I verified the Fourier one and the LLM one. The scaling law one is likely okay too as I long back read the book.

yes, i noticed that occasionally but i'm curious which one did you find is incorrect?

Yeah, that specific one doesn't work so well but apart from it, does any other example not work?

Yes, the skill is something like the following:

# Codex Verification Skill

Use OpenAI Codex as an independent reviewer via `codex exec`.

## How to Call Codex

*Standard pattern with answer extraction:* ```bash CODEX_OUTPUT=$(timeout 120 codex exec '<your prompt here>. Put your complete analysis inside <answer></answer> tags.' 2>/dev/null)

Yeah, all of it was done by Opus 4.6

I read all of the outputs.

Definitions are there if you hover on them.

These are all 256 rules. Where do you spot discontinuities? Also, each rule does show compressibility and other metrics like entropy

I think this shows the future of how agent-to-agent economy could look like.

Take a look at this thread: TIL the agent internet has no search engine https://www.moltbook.com/post/dcb7116b-8205-44dc-9bc3-1b08c2...

These agents have correctly identified a gap in their internal economy, and now an enterprising agent can actually make this.

That's how economy gets bootstrapped!

https://invertedpassion.com - write essays on systems, philosophy, science, tech and startups

Humans actually do both. We learn from on-policy by exploring consequences of our own behavior. But we also learn off-policy, say from expert demonstrations (but difference being we can tell good behaviors from bad, and learn from a filtered list of what we consider as good behaviors). In most, off-policy RL, a lot of behaviors are bad and yet they get into the training set and hence leading to slower training.

Aren't manifolds generally task-dependent?

I've been debating whether the data lies on a manifold, or whether the data attributes that are task-relevant (and of our interest) lie on a manifold?

I suspect it is the latter, but I've seen Platonic Representation Hypothesis that seems to hint it is the former.

Is it fair to say that both "Say 'an'" and "Say 'astronomer'" output features would be present in this case, but say "Say 'an'" gets more votes because it is start of the sentence, and once it is sampled "An" further votes for "Say 'astronomer'" feature

LLMs have induction heads that store such names as sort of variables and copy them around for further processing.

If you think about it, copying information from inputs and manipulating them is a much more sensible approach v/s memorizing info, especially for the long tail (where not enough "storage" might be worth allocating into network weights)

Thanks! Vijaye

Thank you!

Sorry about that! We now have server side testing option where A/B tests trigger from the backend.

Thank you!

Comparison with PG's essays is a great honor. His essays are what inspired me to go down the startup journey myself.