Intellect-2 Release: The First 32B Model Trained Through Globally Distributed RL

https://www.primeintellect.ai/blog/intellect-2-release

by Philpax • 1 year ago

201 65 1 year ago

primeintellect.ai

esafak• 1 year ago

How are they ensuring robustness against adversarial responses?

nsingh2• 1 year ago

From the article, seems like TOPLOC:

based on top of novel components such as TOPLOC, which verifies rollouts from untrusted inference workers

https://github.com/PrimeIntellect-ai/toploc

xmasotto• 1 year ago

Can an expert explain how this protects against adversarial actors?

At a glance it looks like something akin to a computing a checksum that's locality sensitive, so it's robust to floating point errors, etc.

What's to stop someone from sending bad data + a matching bad checksum?

ndgold• 1 year ago

Pretty badass

quantumwoke• 1 year ago

Wonder what the privacy story is like. Enterprises don't usually like broadcasting their private data across a freely accessible network.

bjt12345• 1 year ago

A strong use case here for quantum-safe encryption.

Zambyte• 1 year ago

Why? Quantum safe cryptography is mostly interesting right now in the context of defending against store now, decrypt later attacks. That doesn't seem helpful here, because they'll still need to decrypoit for training. Did you mean homomorphic encryption?

mountainriver• 1 year ago

Awesome work this team is doing. Globally distributed MoE could have real legs

refulgentis4• 1 year ago

I guess I'm bearish?

It's not that they trained a new model, but they took an existing model and RL'd it a bit?

The scores are very close to QwQ-32B, and at the end:

"Overall, as QwQ-32B was already extensively trained with RL, it was difficult to obtain huge amounts of generalized improvement on benchmarks beyond our improvements on the training dataset. To see stronger improvements, it is likely that better base models such as the now available Qwen3, or higher quality datasets and RL environments are needed."

fabmilo• 1 year ago

The interesting delta here is that this proves that we can distribute the training and get a functioning model. The scaling factor is way bigger than datacenters

refulgentis4• 1 year ago

The RL, not the training. No?

comex• 1 year ago

But does that mean much when the training that produced the original model was not distributed?

christianqchung• 1 year ago

Third party fine tuned open weighted LLMs tend to be good at a handful of benchmarks, but parity or lower on others compared to the original model. There are some exceptions like Nvidia's Nemotron series, but the differences generally are so small as to be imperceptible. Deepseek released finetunes of several Qwen and Llama models alongside R1, and while they were better in some select (mostly math) and coding domains, there were problems resulting from fine tuning that didn't result in them overtaking the original models in usage.

cess11• 1 year ago

Seems that's mostly a byproduct from working on the core business idea, GPU arbitrage.

jumploops• 1 year ago

Congrats to the team on the launch!

Personal story time: I met a couple of their engineers at an event a few months back. They mentioned they were building a distributed training system for LLMs.

I asked them how they were building it and they mentioned Python. I said something along the lines of “not to be the typical internet commenter guy, but why aren’t you using something like Rust for the distributed system parts?”

They mumbled something about Python as the base for all current LLMs, and then kinda just walked away…

From their article: > “Rust-based orchestrator and discovery service coordinate permissionless workers”

Glad to see that I wasn’t entirely off-base :)

Havoc• 1 year ago

Given the latencies at play python did probably make more sense though

bwfan1232• 1 year ago

The technical underpinning has nothing to do with the language. It is a different way of optimizing parameters called diloco. I agree though that python is an abomination for systems services componentry when there are languages like rust.

throwanem6• 1 year ago

There's a name and a logo. "Hubris" feels slightly beggared. https://en.m.wikipedia.org/wiki/The_Metamorphosis_of_Prime_I...

Extropy_• 1 year ago

This looks like a startup company. Why shouldn't it have a name and logo?

Philpax2• 1 year ago

Their point is that the name and logo are clearly drawing from the Metamorphosis of Prime Intellect, with all the potential baggage that comes with it. It's an interesting choice.

bcoates• 1 year ago

Maybe torment nexus was taken

schneehertz• 1 year ago

I used to have an idea related to science fiction novels that artificial intelligence could aggregate computing power through the network to perform ultra-large-scale calculations, thereby achieving strong artificial intelligence. Reality will also develop in this way, which is very interesting

abtinf• 1 year ago

Does this have anything to do with The Metamorphosis Of Prime Intellect, or did they just abuse the name and the cover art?

arthurcolle• 1 year ago

Prime Intellect is a grabby AI :)

danielhanchen• 1 year ago

I made some GGUFs at https://huggingface.co/unsloth/INTELLECT-2-GGUF

./llama.cpp/llama-cli -hf unsloth/INTELLECT-2-GGUF:Q4_K_XL -ngl 99

Also it's best to read https://docs.unsloth.ai/basics/tutorial-how-to-run-qwq-32b-e... on sampling issues for QwQ based models.

Or TLDR, use the below settings:

./llama.cpp/llama-cli -hf unsloth/INTELLECT-2-GGUF:Q4_K_XL -ngl 99 --temp 0.6 --repeat-penalty 1.1 --dry-multiplier 0.5 --min-p 0.00 --top-k 40 --top-p 0.95 --samplers "top_k;top_p;min_p;temperature;dry;typ_p;xtc"

3abiton• 1 year ago

This is rather exciting! I see the future of Co-op models made by a community of experts on a specific field that would still allow them to be competitive with "AI monopolies". Maybe not all hope is lost!

iTokio• 1 year ago

It’s interesting that it does something useful (training a LLM) without trust and in a decentralized way.

Maybe this could be used as proof of work? To stop wasting computing resources in crypto currencies and get something useful as a byproduct.

proof_by_vibes• 1 year ago

There could be merit to this. Proofs are generally computationally hard, so it's possible that a currency could be created by quantifying verification.

littlestymaar• 1 year ago

To stop wasting computing resources in crypto currencies and get something useful as a byproduct.

Bitcoin is the only major cryptocurrency that still use proof of work today (others are either using “proof of stakes” or are “Layer 2” chains), and due to its (relative lack of) governance structure, it's very unlikely to ever change.

fastball• 1 year ago

The emphasis is indeed on "without trust" – as far as I can tell this project is unable to verify whether the decentralized training nodes are contributing productively.

Without the ability to validate that training compute is heading in the globally desired direction, it is unlikely you could use it as the foundation of a (sound) cryptocurrency.

mentalgear2• 1 year ago

The reward model could be used as a validation/reward for the client. Give the same nodes the same inferences to make, and the one with the highest reward (those could be short, or even partially calculated long-term) will also get the "currency" reward.

mentalgear2• 1 year ago

That would be indeed a very promising way of FINALLY making cryptocurrency useful!

_ink_• 1 year ago

I read an argument, that proof of work needs to be useless and wasteful. If it would produce value in itself it would make 51% attacks more economic and thus the currency less secure.

throwanem6• 1 year ago

Sure. The whole point of "proof of work" is to show (prove) you've lost energy to heat (work). That's what makes it costly and thus an honest signal.

The model breaks where work can be counterfeited (usually impossible) or where energy prices go to zero, which is why "bitcoin colonialism" was briefly a thing last decade. Much of bitcoin's design, this aspect also, is intended to protect against the bare-fanged, red-eyed money weasels it was also designed to attract.

ucha• 1 year ago

It needs to not have economic value but it doesn't necessarily need to be useless and wasteful.

api2• 1 year ago

I’ve seen an argument that military power and credible threat are the proof of work mechanism for fiat currencies. That is also useless, but it does throw off secondary useful effects like inventions.

Not totally convinced the analogy maps but interesting.

Geee• 1 year ago

No, this process doesn't produce "proof of work", i.e. verifiable proofs that energy has been used.

naasking3• 1 year ago

New weights that have lower loss than the input weights is proof that work has been done.

k__• 1 year ago

Arweave and Filecoin use PoW algorithms that prove something useful.

bastawhiz3• 1 year ago

Maybe this could be used as proof of work

There's nothing provable here. Crypto proof of work is easily verified (does the hash of this value look the way I expect?). How do you prove in ~O(1) time that someone did some operation with their GPU? You don't. You don't even know what the thing is that you're training (without a trained model you don't have the ability to know whether the model the was allegedly trained learned the thing you want it to learn).

naasking3• 1 year ago

How do you prove in ~O(1) time that someone did some operation with their GPU? You don't.

The work in this case could be that the weights after the was done work have lower loss than the input weights. Applying the new weights to input to check that it's lower is much cheaper than calculating the weights, which is the same trend as proof of work (not sure about the magnitude of difficulty being enough to replace proof of work though).

Thomashuet• 1 year ago

Summary: We've use the most complexest, buzzwordiest training infrastructure to increase the performance of our base model by a whopping 0.5% (±1%).

Weryj• 1 year ago

But this isn’t about the performance, the infrastructure is the product here.

lonelyasacloud• 1 year ago

Indeed, most reliable way to make money in a gold rush is to sell shovels.

Mougatine• 1 year ago

very cool work!

bwfan1232• 1 year ago

The most interesting thing I see is the productization of the diloco work done here [1]. If someone can make this scale, then we can say goodbye to expensive backend networking and mainframe-like AI training machinery.

[1] https://arxiv.org/abs/2311.08105

ikeashark• 1 year ago

I wonder why they randomly noted a torch-compile vs non torch-compile figure where torch-compile degraded model performance. What made it degrade? It seems to only appear in one figure and nowhere else.

Intellect-2 Release: The First 32B Model Trained Through Globally Distributed RL

Related Stories