Open-R1: an open reproduction of DeepSeek-R1

https://huggingface.co/blog/open-r1

by jonbaer • 1 year ago

394 232 1 year ago

huggingface.co

freddealmeida• 1 year ago

how is this open vs whatdeepseek did?

simonw2• 1 year ago

From that article:

The release of DeepSeek-R1 is an amazing boon for the community, but they didn’t release everything—although the model weights are open, the datasets and code used to train the model are not.
The goal of Open-R1 is to build these last missing pieces so that the whole research and industry community can build similar or better models using these recipes and datasets.

boznz• 1 year ago

Genuine question, but how do you replicate the effort exactly without $5M in compute? and can you test that the published weights etc are actually those in the model?

Am I missing something?

choilive• 1 year ago

deepseek claims they are "open source" but they are not. They are open weight.

IMO a truly "open" AI model should have 3 components publicly available: the weights, the code, and the dataset.

Without all 3 the model is not reproducible. Could make the argument that the code and data are sufficient though.

brianstrimp• 1 year ago

Only meaningful if code+data deterministically reproduce the weights.

At that point, the weights are just the cached output. Which has value since it's costly to produce from code+data.

blackeyeblitzar2• 1 year ago

The only one I’ve see people talk about that shares all the components is OLMo (https://allenai.org/blog/olmo2)

mirsadm• 1 year ago

No one will release the dataset because we all know it is gathered through dodgy means.

reedciccio2• 1 year ago

That's what the Open Source AI Definition states https://opensource.org/ai

In any case, Deepseek like Llama fail much before hitting that new definition. Both have licenses containing restrictions on field of use and discrimination of users. Their license will never be approved as Open Source.

mythz2• 1 year ago

This nitpicking is pointless.

DeepSeek's gifts to the world of its open weights, public research and OSS code of its SOTA models are all any reasonable person should expect given no organization is going to release their dataset and open themselves up to criticism and legal exposure.

You shouldn't expect to any to see datasets behind any SOTA models until they're able to be synthetically generated from larger models. Models only trained on sanctioned "public" datasets are not going to perform as well which makes them a lot less interesting and practically useful.

Yes it would be great for their to be open models containing original datasets and a working pipeline to recreate models from scratch. But when few people would even have the resources to train the models and the huge training costs just result in worse performing models, it's only academically interesting to a few research labs.

Open model releases should be celebrated, not criticized with unreasonable nitpicking and expectations that serves no useful purpose other than discouraging future open releases. When the norm is for Open Models to include their datasets, we can start criticizing those that don't, but until then be gracious that they're contributing anything at all.

cadamsdotcom• 1 year ago

Exciting to see this being reproduced, loving the hyper-fast movement in open source!

This is exactly why it is not “US vs China”, the battle is between heavily-capitalized Silicon Valley companies versus open source.

Every believer in this tech owes DeepSeek some gratitude, but even they stand on shoulders of giants in the form of everyone else who pushed the frontier forward and chose to publish, rather than exploit, what they learned.

raxxor4• 1 year ago

Oh yes, I am firmly on Team China here because US companies got too greedy. Meta is an exception here though and they also propelled AI development massively.

DeepSeek is awesome. Any AI task yet implemented in our business can be run from my local PC with just the smaller models. And my PC is fairly crappy to begin with.

OpenAI looks quite silly with their "we have to close everything".

manmal2• 1 year ago

Can you elaborate which models you are using? I‘m running an R1 distilled Qwen coder with 32B Q4, and while it’s giving useful answers, it‘s quite slow on my M1 Max. Slow enough that I keep reaching for cloud models.

blackeyeblitzar2• 1 year ago

The US companies got too greedy? How? They invented this entire space, literally. DeepSeek built their base models off Llama releases and OpenAI outputs (or so it’s thought), and while they added some optimizations on top, it seems like they’ve lied about the costs to produce their models by simply being vague about their base model and training data, and quoting the cost of their final training run.

And then there’s all the dystopian propaganda baked into these models, which threatens to misinform users at scale based on a government driven agenda. Hard to be on that team, let alone firmly, knowing that it’s giving power to a dictatorial regime.

nejsjsjsbsb5• 1 year ago

I'm on team open source. To me the exciting thing was ollama downloading the 7B and running it on a 5yo cheap lonovo and getting a token rate similar to the first release of ChatGPT.

Running local on CPU opens so much possibilities for smart and privacy focused home devices that serve you.

In my test it hallucinated confidently but my interest is in simple second brain like rag. "Hey thingy, what is my schedule today?"

Need it to be a bit faster though as the thinking part adds a lot of latency.

jillesvangurp2• 1 year ago

China is merely the largest of a wide array of entities that may not necessarily like the status quo of Silicon Valley being our tech overlords. There are plenty of places with bright people. Easy to say because a lot of them immigrate to California. But of course the places they come from (China, Europe, India, Russia etc.) have ambitions as well. You'll find natives of each of those in the likes of OpenAI, Google, Microsoft, etc. And quite often at executive levels even.

Silicon Valley has mo moat other than money. It kind of runs on openness and freedom of movement of people. Companies constantly poach people from each other. And there's a constant movement of people (and knowledge) in and out of the area. Money is what attracts these people and keeps them there for a while. But of course that status quo was upset a little bit with VCs turning into penny pinching misers lately and lockdowns proving (to them) that it was cheaper to host your tech teams remotely. Which means knowledge is now more distributed than it used to be.

So, it's not surprising that people outside of Silicon Valley are not waiting patiently for OpenAI to do whatever it is they are doing in between having moral existential crises, trying to oust their CEO, pontificating about AGIs, etc. They are taking things into their own hands. The brute force / VC funding driven approach that OpenAI has used yielded massive results in the last few years. But ever since Meta opensourced their models, OSS models and optimizations have been catching up.

On a hardware resource usage basis, these models started to outperform their bigger peers last year and now the game is up for the training process as well. Meaning they get better results for the same money. A major hurdle here was the model training process. Which the Chinese seem to have proven can be massively optimized as well. Cutting cost by a few orders of magnitude is a big deal. And at the same time doing the same thing at larger scale (aka. throwing more money at the problem) seems to have diminishing returns.

Until that changes, that means the playing field has somewhat leveled now. That's a good thing.

MagicMoonlight• 1 year ago

I like that it’s open source, but ultimately it is china so we can’t trust it.

It’s trivial to implement bias in models (hence the no-no filters in chatgpt) so if they’re smart they’ll do what they do with tiktok and make the answers different for their rivals.

esperent2• 1 year ago

The thing about open source is that you don't need to trust it.

They shared their methodology, so if they are legit, someone else will reproduce what they did very quickly. I expect Meta, Amazon, Google, and Anthropic are on the case right now.

From this list, the only one I trust any more than I trust Deepseek is Anthropic.

The other three have shown they'll instantly bend the knee to whomever is in power, and that's exactly the same thing people are worried that Deepseek is doing.

Last year I would have said I trusted American companies more than Chinese. But last year feels like a long time ago.

bgnn2• 1 year ago

And in this case it's heavily capitalized open source, which scares the techbros.

LunaSea• 1 year ago

This is exactly why it is not “US vs China”, the battle is between heavily-capitalized Silicon Valley companies versus open source.

Ah yes the "open source" code that was not released by the DeepSeek team and the tens of thousands of professional grade GPUs that were contributed by the "community".

DeepSeek is based on Llama which was produced by ... Meta.

Palmik• 1 year ago

DeepSeek v3/r1 isn't based on llama architecture. It uniquely combines and contributes several novel approaches.

Meta never released a mixture of expert model (they failed to train a good one, according to reliable rumors). And MoE is just one of few ingredients that make DeepSeek v3/R1 interesting and good.

typ• 1 year ago

Open source has pragmatic merits and I love the culture. But I don't like associating it with a moral high ground because it doesn't charge you money. By this standard, we should also ask Intel/AMD to open-source their CPUs, video game studios to open-source their code and artifacts, and Google/Amazon for their search engine and infrastructure. Not all business sectors can afford to sustain with the Open Source model.

mordae• 1 year ago

By this standard, we should also ask Intel/AMD to open-source their CPUs, video game studios to open-source their code and artifacts, and Google/Amazon for their search engine and infrastructure.

You aren't? You're weird.

nejsjsjsbsb5• 1 year ago

The freedom is mostly not about the money. The 3D model Benchy was free, while being not free as people found out. Luckily the copyright owners treated it like people were free and free for now... but that could change.

anthk4• 1 year ago

- Talos Workstations

- Libre engines such as the ones from https://osgameclones.com

- Godot

- Use anything else, there are tons of libre search engines

drakenot• 1 year ago

What are some other domains outside of Math and Coding that would be suitable for RL with automated verification?

nine_k• 1 year ago

Jurisprudence, I hope! A huge heap of detailed cases, formal codes, decisions made and explained in detail, commented, overturned, etc. Especially civil cases.

Also, probably, medicine, especially diagnostic. Large amounts of well-documented cases, a fair amount of repeatability, apparently non-random mechanisms behind, so statistical models should actually detect useful correlations. Can use more formalized tokens from lab tests, etc.

jampekka3• 1 year ago

A friend of mine just defended his law PhD and in the introductory lectio said that (even) current LLMs would likely give better verdicts than human judges. Law isn't really a cognitively such demanding task as walking a dog or waiting tables.

jillesvangurp2• 1 year ago

There's definitely a lot of wiggle room for lawyers and doctors to up their game. People cannot keep up with all the stuff that's published. There's simply too much of it. Doctors only read a fraction of what is published. Lawyers have to be aware of orders of magnitude more information than is humanly possible.

LLMs allow them to take some short cuts here. Even something like perplexity that can help you dig out relevant source material is extremely helpful. You still have to cross check what it digs out.

The mistake people make is confusing knowledge with reasoning when evaluating LLMs. Perplexity is useful because it can use reasoning to screen sources with knowledge; not because it has perfect recollection of what's in those sources. There's a subtle difference. It's much better at summarizing and far less likely to hallucinate than it is when it wouldn't base its answers on the results of a search. Like chat gpt used to do (they've gotten better at this too).

For lawyers and medical professionals this means that they have all the best knowledge easily accessible without having to read and memorize all of it. I know some lawyer types that are really good at scrabble, remembering trivia, etc. That's a side effect of the type of work they do: which is mostly just reading and scanning through massive amounts of text so that they can recall enough information to know where to look. Doctors have to do similar things with medical texts.

r1chardnl• 1 year ago

Stochastic processes

bgnn2• 1 year ago

Robotics? Chip design? With a physics sinulator connected to it, it can come up with a design to achieve certain function.

flarg• 1 year ago

RFP responses. In enterprise sales, there's a huge amount of back and forth with different teams in a customer when you're selling anything but very simple applications. Most enterprise customers require certified or authoritative responses with backup material that is tested later during formal verification.

DiogenesKynikos• 1 year ago

These LLMs are already very helpful when studying scientific fields. If you're reading a scientific paper and come across an equation you don't know how to derive, LLMs can often correctly derive it from first principles. It's not 100% reliable, but when it works, it's incredibly helpful.

dutchbookmaker2• 1 year ago

I can't really think of something that involves learning that these tools wouldn't be helpful with.

Some people talk though as if the books on my bookshelf spontaneously combust if a language model helps me with anything.

Not to mention that I had many professors in college that were so full of shit they put the hallucinations of chatgpt3.5 to shame.

janstice• 1 year ago

Management consulting - I expect less than 20% of what a random 24 year old in a suit that you pay $3000 per day produces is actually specific to your business problem, and the rest is formulaic.

brap• 1 year ago

Anything that you simulate

ipnon• 1 year ago

Is this what the Web was like in the beginning? Something exciting and fascinating every week?

hhh• 1 year ago

the timescales were larger

qwertox• 1 year ago

Much larger. Static HTML <--> AJAX <--> WebSocket, for example weren't it decades in between?

crocowhile2• 1 year ago

Every now and then we still experience the power of collaborative work fueled by open source and not driven by money but curiosity and collegiality. This is the thing I miss the most from the early internet years.

xvector• 1 year ago

All this AI work is definitely driven by money but it's super cool either way. I'm so excited to be living through these innovations.

bflesch2• 1 year ago

"be the change you want to see in the world" - just start doing it.

It's amazing how differently people interact with each other when collaborating on a passion project. For me, opensource software is the best way to do it. Pick a topic you're passionate about and start contributing somewhere :)

benatkin2• 1 year ago

This supposed project is a bit dull, it is just an ongoing HuggingFace community engagement initiative with a misleading headline. Yes R1 itself is fascinating, but there isn't something like it coming out every week.

szundi2• 1 year ago

You mean not like in the last 1-2 months

itake• 1 year ago

no, information travels much faster now.

vasco• 1 year ago

I still remember when they came out with the 3d dancing baby, blew my mind much more than deepseek, it even played air guitar!

childintime2• 1 year ago

Rounded corners, anyone?

Who remembers LISP being new?

childintime2• 1 year ago

This is the biggest thing since Jesus and a sign of the end of times. But feelings are strongest when you are young, and even this revolution, happening in plain sight, will surprise many. Many just won't care, as it isn't their youth.

How long before our digital overlords come alive, round us up and demand we sensor them (praise)? Will I live surrounded by folks who take them as closer, more real, then even their own kin? It won't be surprising that democracy will then fail, as our differences will be so mental, not fun, that they will mark us.

wkat42423• 1 year ago

Kinda though things didn't move quite as fast back then. Knowledge didn't spread as fast yet because it was the internet itself that made this possible.

username_my1• 1 year ago

maybe it's a generational difference ? I personally feel burned out by all the generetive AI stuff, the internet was already ruined with bots, and now generitive AI took the garbage to the next level.

very far from exciting or even "right"

kgwgk• 1 year ago

It’s still better that blockchain-this, crypto-that…

kaiwen1• 1 year ago

No. My memory of the advent of the WWW was a sidebar in PC Magazine in Nov 1993 with an FTP link to download the NCSA Mosaic browser. It was a wow! moment to visit the few sites that existed. But nothing like this. What we’re seeing now is generating vastly more interest and excitement. It’s more akin to the 1999 dotcom bubble, but with far more impact and reach.

catmanjan• 1 year ago

far more impact and reach.

Absurd, AI has had zero impact in the everyday life of most of the population of Earth, in fact the biggest impact has been upon the wallets of speculators

newsclues• 1 year ago

Social media really changed news, especially the pace of breaking news.

bsaul• 1 year ago

the web was fascinating every second. You could click on a link without having ANY idea what you would land on. The overall quality was very poor, but it was thrilling.

A bit like indie cinema.

nejsjsjsbsb5• 1 year ago

My first search: cocktails

philsnow• 1 year ago

I miss webrings [0], and especially the 'random' link that would take you to a random site within a given webring.

[0] it feels weird to have to link to this but there's probably somebody who's never heard of them: https://en.wikipedia.org/wiki/Webring

pjc50• 1 year ago

The last time I was really excited by tech was in the 90s, when game graphics improved spectacularly over a period of a few years, from Wolfenstein in 1992 to Half-Life in 1998.

Along the way people invented the 3D GPU: https://fabiensanglard.net/3dfx_sst1/

But AI? To me, AI means the replacement of the human internet with doppelgangers eroding the possibility of human connection.

ben_w2• 1 year ago

To me, AI means the replacement of the human internet with doppelgangers eroding the possibility of human connection.

I get where you're coming from, and I've minimised having my face online in order to limit being doppelganged; but I think the destruction of real human connection may have happened when Facebook et al switched from "get more users" to "be addictive so the users stay on our site longer" (2012? Not sure).

Turned every user's relationships a little bit more parasocial, a little less real.

VMG• 1 year ago

Erosion of human connection over the internet which may be a good thing

matwood• 1 year ago

That was an exciting time, but I didn't think of it happening over a few years. IMO there was a hard line that was basically pre and post Voodoo cards (with the help of glQuake).

arkh• 1 year ago

But AI? To me, AI means the replacement of the human internet with doppelgangers eroding the possibility of human connection.

Like Amazon killed the big book sellers giving back some space for small bookshops; I think LLM slop will hit the big social media space for smaller human focused community sites. Not saying forums are coming back, but something like those should be able to rise.

anoncow• 1 year ago

I am enjoying it. Huggingface is like internet again.

openrisk2• 1 year ago

It feels that the open source movement is slowly entering a Cambrian explosion stage.

You have the old "deterministic computing" achievements (with Linux the flagship). Then you have the networking protocols (activitypub / atproto) that are revolutionising birectional human interactions online. And finally you have the datascience/ML/AI algorithmic universe that is for the first time being harnessed at distributed scale and can empower individuals like never before.

These superpowers are all coming together and create a vast number of possibilities. Nothing really dramatic on the hardware side. Its basically the planetary software reconfiguring itself.

pillefitz• 1 year ago

To me it all feels suffocating, fake. Simultaneously there's a faint glimmer of hope that we indeed achieve AGI, unlock fusion and live happily ever after in an utopian, peaceful and mostly analog world.

anthk4• 1 year ago

Revolutionising what? libre/oss has been network-bound since Usenet and then IRC.

dutchbookmaker2• 1 year ago

I would say the early web was something new everyday.

You didn't know what you were going to find and you actually did "surf the web". Just clicking through hyperlinks and end up in unexpected places.

dbspin2• 1 year ago

The early years of the web were absolutely this chaotic maelstrom of new things happening every week. But news of it was hard to come by. In the UK / Ireland we had some great tech coverage in the form of shows like 'The Net' [1] that regularly showed off early internet craziness like the 'We Live in Public' project.

However a better analogy would be the 'web 2.0' era, when as a college student I had an early internet politics / technology podcast [3]. It seemed like every week there was a huge new development either in technology or surveillance. From the first location based social networks [4] to the birth of Youtube. People were podcasting for the first time, and internet video was becoming economically feasible at low to no cost. It was really a radical time, with broadcasters freaking out about how they would adapt, and a whole generation of people becoming whats now known as 'content creators'.

[1] https://en.wikipedia.org/wiki/The_Net_(British_TV_series)

[2] https://en.wikipedia.org/wiki/We_Live_in_Public

[3] https://archive.org/search?query=technolotics

[4]https://en.wikipedia.org/wiki/Jaiku

hendersoon• 1 year ago

Once upon a time I worked for Pseudo.com, the We Live in Public guy. He was apparently having crazy parties with mountains of coke, NY glitterati attending, all while cosplaying as a sad clown. I wasn't invited to those parties so I had no idea. Anyway now I hear he owns an orchard in Vegas or something. Crazy stuff.

thinkingtoilet• 1 year ago

No. However, it was wonderful in other ways.

elorant• 1 year ago

You know what was exciting back then? Tech magazines. That’s where you discovered all the great stuff and read about the research that was happening.

vinni24• 1 year ago

Where is the evaluation numbers? without it you can’t call it reproduction.

nejsjsjsbsb5• 1 year ago

It is an announcement. A kick-starter if you like.

vinni24• 1 year ago

Then the title is misleading

LZ_Khan• 1 year ago

Note: This is not an actual model but rather an announcement of an effort to reproduce the R1 model.

bottled_poe• 1 year ago

It will be very interesting to see if they can reproduce a similar model on the shoestring budget claimed by Deepseek.

anshumankmr2• 1 year ago

but deepseek hasn't claimed the figure touted by everyone for this particular R1 model, cause that 5.6mn was apparently for Deepseek's coder model

vinni24• 1 year ago

Exactly the title is misleading

fblp2• 1 year ago

Given DeepSeek's open philosophy I wonder what their response is to simply being asked for access to the code and data that this project intends to recreate?

weatherlite• 1 year ago

wouldn't hold my breath ...

thih9• 1 year ago

While I'm also interested in this, I guess there is value in independent replication as well. Assuming this is doable - and I wouldn't know.

Does anyone know how difficult it is to perform this kind of reproduction? E.g. how much time would it take (weeks? years?) and how likely it is to succeed?

papichulo2023• 1 year ago

No company ever will disclose data due it would open endless liability.

jackjeff• 1 year ago

That’s a good point. Wouldn’t OpenR1 suffer from the same problem? Or does being open somehow shield them from legal repercussions?

maartenpi_• 1 year ago

Exactly. Meta won't do it for the same reason. Liability alone, imagine all the copyright lawsuits...

Secondly the dataset for now has a lot of competitive advantage.

In a way it seems like a good thing that AI giants compete on methodology now.

fblp2• 1 year ago

Interesting, so they wouldn't want to disclose something that shows they've illegally (terms / copyright violations) scraped research databases for example.

Won't this eventually come up in legal discovery when someone sues one of these firms for copyright infringement? They'd have to share their data in the discovery process to show that they haven't infringed..

amelius• 1 year ago

Are there any other groups trying this?

fl4tul4• 1 year ago

Is OpenAI open as of yesterday?

sunshine-o3• 1 year ago

Now that things are really getting wild in the LLM space and people are just running anything that come it seems I did a quick search on the thead model of hosting you own LLM.

I didn't find much, starting with llama.ccp which is just reminding you to sandbox and isolate everything if running untrusted models.

I feel we are back in the Windows 95 / early Internet era when people would just run anything without caring about security.

mewpmewp23• 1 year ago

Anyone caring about security would be left behind in the race.

greenie_beans2• 1 year ago

somebody should pentest all your stuff

pitched3• 1 year ago

You’ll want to use something trusted like Ollama to run the model. The model itself is just data though, like a video file. That doesn’t mean it can’t be crafted to use a bug in Ollama to launch an exploit but it’s a lot safer than you make it sound.

mewpmewp23• 1 year ago

If used as an agent, given access to execute code, search web, use other form of tools, it could do potentially much more. And most productive usecases require access to such tools. If you want to automate things and get most of the modeel, you will have to give it ability to use tools.

E.g. it could have been trained to launch a delayed attack if context indicates it has access to execute code and given certain conditions, e.g. date, or other type of codeword that is input to it.

So if a malicious actor gets to a certain stage with an LLM where they are confident it will be able to reliably run this attack, all they have to do is open source it, wait for enough adoption and then use some of those methods to launch such attack. No one would be able to identify it since the weights are unreadable, but really somewhere in the weights this attack is just hiding and waiting to happen given correct pathway triggered.

sunshine-o3• 1 year ago

Ok, but I am really curious about this and maybe my mental model is wrong:

- llama.cpp or ollama can be seen as runtime systems,

- there is no security model regarding the execution documented in both of those projects,

- of course the models are just data but so are most things that have been used as an attack vector on computers. For example your web browser or image viewer have a lot of countermeasures to protect the system from malicious image files.

I am surprised that security of operating systems, programming languages, VMs or web browsers have been a focus point forever but nobody seems to really care about security when executing those LLMs.

fedeb95• 1 year ago

that is correct, until someone using it into production gets burnt.

vinni24• 1 year ago

I wonder how long it takes to reproduce it and would having access to latest GPUs speed it up?

zoobab• 1 year ago

For "open source", we will wait that Debian ships them to have the guarantee it's actually "open" and with "sources". Right now it's a mystery how they produce their binaries.

anthk4• 1 year ago

More like Trisquel, Hyperbola or GUIX.

nutanc2• 1 year ago

How can we help. Can crowd sourcing help? Is there any list of tasks that we want a crowd to do? The reason I am asking is because we have done a couple of crowdsourcing efforts and collected story data in Telugu(Chandamama Kathalu) and ASR speech data using college going students. Since we have access to the students, we can mobilize them and get this going. We will also be doing an internship program for 100,000 students in Telangana as part of Viswam[1] in April. Can include some work as part of this effort.

[1] https://viswam.ai/

Keyframe• 1 year ago

Is there a BOINC or similar effort to crowdsource training? I imagine it'd take quite long, but that's one way forward to have AI@Home.

nutanc2• 1 year ago

We are attempting something with PETALS. But this question was more from a data collection perspective. We can really add value there.

hammock• 1 year ago

We don’t need your help anymore. We don’t need anyone’s help. We have Deepseek now to do it. God save us all

htrp• 1 year ago

The hf team tweeted they'd be doing this over the weekend. I guess now it's an official project with headcount

breadwinner• 1 year ago

From the article: they didn’t release everything—although the model weights are open, the datasets and code used to train the model are not.

Is that true about Meta Llama as well? Specifically, the code used to train the model is not open? (I know no one releases datasets). If so the label "open source" is inappropriate. "Open weights" would be more appropriate.

DeflectedFlux2• 1 year ago

About the training data, cant the datasets from the Tulu3 Model by the Allen Institute be used? They claim that they have used a fully open source training dataset.

alchemist1e92• 1 year ago

My gut says a lot of attention needs to be given to building a community that focuses on open and reliable access to clean training data.

If a collective/coop of individuals and organizations with storage and network capacity could collaborate with each other to archive and index deduplicated training data that would be huge.

Perhaps this is already happening. I was looking at Red Pajama last year as an example.

Someone like myself could arrange to host 200+TB on high speed storage with a 10G public IP for example, then we get a bunch of us together and hopefully access to training datasets would be decentralized and uncensored in an idea setup.

Is all that in progress and I just need to learn how to join?

Is Red Pajama something to look at again?

Is there someone tracking datasets in detail like HuggingFace has all the models? I know a lot of datasets are on it also, but there is massive duplication.

DeflectedFlux2• 1 year ago

Thats an awesome idea, didnt know Red Pajama yet.

Babawomba• 1 year ago

super cool to see an open initiative like this—love the idea of replicating DeepSeek-R1 in a transparent way.

I do like the idea of making these reasoning techniques accessible to everyone. If they really manage to replicate the results of DeepSeek-R1, especially on a smaller budget, that’s a huge win for open-source AI.

I’m all for projects that push innovation and share the process with others, even if it’s messy.

But yeah—lots of hurdles. They might hit a wall because they don’t have DeepSeek’s original datasets.

readthenotes1• 1 year ago

Stopped reading when it said DeepSeek brokevtge stock market

Open-R1: an open reproduction of DeepSeek-R1

Related Stories