jxmorris12

Me too. maybe Pangram solves this?

Why did Qwen stop producing open models? They've gone from building the best open models ~1 year ago to producing like the 10th-best closed models. I don't understand this pivot at all.

Edit: I saw online they do in fact plan to release this openly at some point – x.com/Alibaba_Qwen/status/2078759124914098291

Lo and behold, a nice arithmetic coding implementation that wasn't written by an LLM! A sight for sore eyes – a treat, even. Looks like it was written by someone else though.

Check it out: https://github.com/samyak112/pym-particles/blob/main/arithme...

Haha. Unfortunately is my regular voice, since long before I started using Codex. You can check through some of my old writing. It definitely could've gotten worse though. Not sure if I'm training on Codex, or Codex is training on me...

This is certainly part of it! My point was that focusing on problems proposed by others is one very specific and pretty short-term mode of thinking. Good researchers improve benchmark scores. Great researchers think about what problem they're solving.

Incredible concept and a very well-crafted site. I scored very low, but then very high with my legal name. It seems DeepSeek knows a lot of arxiv papers (or at least, about the authors).

There’s nothing to read.

Model A: A_1, …, A_n Model B: B_1, …, B_n

C_i = A_i * p + B_i * (1 - p)

In other words, it’s just a linear combination of the other models’ weights, per position.

The app does absolutely no work in the background. It works by simply existing as a running process, thanks to having the same bundle identifier as the Music app.

I love clever, low-or-no-code engineering solutions like this. You typically need to understand a systems very deeply to reach this level of elegance. In this case, one has to understand exactly what happens when the play button is pressed in Mac OS, how bundle identifiers work, etc. And the outcome is an app with almost no code at all – just a collision – it's beautiful.

(As an aside, coding agents are terrible at this kind of thing; I'd guess Codex as of right now would write some overpowered application that polls in a loop looking for Music App starts and killing them)

For some reason I can't get past the start page; I press all the buttons but nothing happens. Rats.

Which one? San Francisco right now is "Untitled Brown and Gray"...

I also wonder why this article was flagged. The article is a highly-respected researcher and professor at CMU. His thoughts are worth reading.

Keep typing.

It's great that people are starting to take continual learning seriously, and it seems like Jessy has been thinking about LLMs and continual learning longer than almost anyone.

I especially like this taxonomy

I think of continual learning as two subproblems:

Generalization: given a piece of data (user feedback, a piece of experience, etc.), what update should we do to learn the “important bits” from that data?

Forgetting/Integration: given a piece of data, how do we integrate it with what we already know?

My personal feeling is that generalization is a data issue: given a datapoint x, what are all the examples in the distribution of things that can be inferred from x? Maybe we can solve this with synthetic datagen. And forgetting might be solvable architecturally, e.g. with Cartridges (https://arxiv.org/abs/2506.06266) or something of that nature.

Matryoshka embeddings are not sparse. And SPLADE can scale to tens or hundreds of thousands of dimensions.

It seems to be an error with the classifier. Sorry everyone. I probably shouldn't have posted that graph; I knew it was buggy, I just thought that the Perl part might be interesting to people.

Here's a link to the model if you want to dive deeper: https://huggingface.co/philomath-1209/programming-language-i...

Hi again. I had already written about this later in my blog post (which is unrelated to this thread), but the point was that RLHF hadn't been applied to language models at scale until InstructGPT. I edited the post just now to clarify this. Thanks for the feedback!

Whoops. I hope you can overlook this minor logical error.

I don't think architecture matters. It seems to be more a function of the data somehow.

I once saw a LessWrong post claiming that the Platonic Representation Hypothesis doesn't hold when you only embed random noise, as opposed to natural images: http://lesswrong.com/posts/Su2pg7iwBM55yjQdt/exploring-the-p...

I recently wrote a post about scaling RL that has some similar ideas:

How to Scale RL to 10^26 FLOPs (blog.jxmo.io/p/how-to-scale-rl-to-1026-flops)

The basic premise behind both essays is that for AI to make another big jump in capabilities, we need to find new data to train on.

My proposal was reusing text from the Internet and doing RL on next-token prediction. The linked post here instead suggests doing 'replication training', which they define as "tasking AIs with duplicating existing software products, or specific features within them".

My bad. Do they make this for Chrome?

Yes, you can definitely convert the outputs from one model to the space of another, and then use them.

Hey, I appreciate the perspective. We definitely should cite both those papers, and will do so in the next version of our draft. There are a lot of papers in this area, and they're all a few years old now, so you might understand how we missed two of them.

We tested all of the methods in the Python Optimal Transport package (https://pythonot.github.io/) and reported the max in most of our tables. So some of this is covered. A lot of these methods also require a seed dictionary, which we don't have in our case. That said, you're welcome to take any number of these tools and plug them into our codebase; the results would definitely be interesting, although we can expect the adversarial methods still work best, as they do in the problem settings you mention.

As for the name – the paper you recommend is called 'vecmap' which seems equally general, doesn't it? Google shows me there are others who have developed their own 'vec2vec'. There is a lot of repetition in AI these days, so collisions happen.

Yeah, we didn't get around to testing what the impact would be of having a small amount of aligned data. I've seen other papers asserting that as few as five pairs is enough to go a long way.

Right, that's why the baselines here come from the land of Optimal Transport, which looks at the world through isomorphisms, exactly as you've suggested.

The GAN works way better than traditional OT methods though. I really don't know why, this is the part that feels like magic to me.

Very fair!

Does this extend to being able to analytically determine which concepts are encodable in one embedding but not another? An embedding from a deft tiny stories LLM presumably cannot encode concepts about RNA replication.

Yeah, this is a great point. We're mostly building off of this prior work on the Platonic Representation Hypothesis (https://arxiv.org/abs/2405.07987). I think our findings go-so-far as to apply to large-enough models that are well-enough trained on The Internet. So, text and images. Maybe audio, too, if the audio is scraped from the Internet.

So I don't think your tinystories example qualifies for the PRH, since it's not enough data and it's not representative of the whole Internet. And RNA data is (I would guess) something very different altogether.

Assuming that is true. If you can detect when you are trying to put a square peg into a round hole, does this mean you have the ability to remove square holes from a system?

Not sure I follow this part.

Yes. So the idea was that an orthogonal rotation will 'encrypt' the embeddings without affecting performance, since orthogonality preserves cosine similarity. It's a good idea, but we can un-rotate the embeddings using our GAN.

Yes! For now just our Twitters: - Rishi, the first author: x.com/rishi_d_jha - Me: x.com/jxmnop

And there's obviously always ArXiv. Maybe we should make a blog or something, but the updates really don't come that often.

Given the demonstrated risk of information leakage from embeddings, have you explored any methods for hardening, obfuscating, or 'watermarking' embedding spaces to resist universal translation and inversion?

No, we haven't tried anything like that. There's definitely a need for it. People are using embeddings all over the place, not to mention all of the other representations people pass around (kv caches, model weights, etc.).

One consideration is that's likely going to be a tradeoff between embedding usefulness and invertability. So if we watermark our embedding space somehow, or apply some other 'defense' to make inversion difficult, we will probably sacrifice some quality. It's not clear yet how much that would be.

Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible:

1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit of paired data

2. A vector database startup founder who told me about his plan to randomly rotate embeddings to guarantee user security (and ignored me when I said it might not be a good idea)

The practical takeaway is something that many people already understood, which is that embeddings are not encrypted, even if you don't have access to the model that produced them.

As one example, in the Cursor security policy (https://www.cursor.com/security#codebase-indexing) they state:

Embedding reversal: academic work has shown that reversing embeddings is possible in some cases. Current attacks rely on having access to the model [...]

This is no longer the case. Since all embedding models are learning ~the same thing, we can decode any embedding vectors, given we have at least a few thousand of them.

I've never seen someone write "&" as a total replacement for "and". I don't like the way it makes me feel.