Why DeepSeek had to be open source

Yeah, I would want to double check and confirm they're using safe serialization methods at the very least before using the weights from any model released by a Chinese entity

marcodiego• 1 year ago

Isn't DeepSeek simple/small enough you can run it locally?

deepsquirrelnet• 1 year ago

At least a TB of VRAM to load it in fp16. They distilled to smaller models, which do not perform as well, but can be run on a single GPU. Full R1 is big though.

zbendefy3• 1 year ago

No, the full R1 model is ~650GB. There are quantized version that quantize it down to ~150GB.

What you can run locally are the distilled models, that is actually LLama and Qwen weights further trained on R1's output

nexus_six2• 1 year ago

I've done this (not thoroughly by any means) with OpenSnitch on the Ubuntu machine I have ollama installed on running the 32b R1 weights. No network traffic.

I'm not entirely sure if it is possible to do some type of code execution like that in just the weights themselves, though someone else who knows a bit more about this can weigh in here.

ru5522• 1 year ago

It's been confirmed to run on a machine with no internet access. So it isn't reliant on external requests, though it could still be trying to make them.

apples_oranges• 1 year ago

“We Have No Moat, And Neither Does OpenAI” - Google

jszymborski• 1 year ago

In fairness, it's at least a $5.6M moat at the moment, which is not exactly "no moat" but it is demonstrably not an insurmountable one, and it might become more shallow yet with time.

dauertewigkeit• 1 year ago

It ironically seems like a very similiar market to internet search. There was no moat there either, other than the capital needed to bankroll a better search engine. A lot of these AI companies will eventually fail (not because their models will be significantly worse but because of failure to commercialize), the market will consolidate with only a couple of players (maybe two in the US, one in China and maybe one in Russia). And once that happens the idea of raising enough capital and building a competitive AI company will seem impossible. Exactly like what transpired with internet search after Google won most of the market.

fourside• 1 year ago

Google did absolutely have a moat on internet search. It wasn’t just about bankrolling a an alternative as Microsoft proved time and time again.

mjburgess• 1 year ago

Oof, no -- it's quite the opposite, much to the likely collapse of google in the future.

Holding exabytes of data to be processed on commodity hardware to enable internet-wide search, all the while it was man-in-the-middle monetised by an ad-business, created tremendous moats. Entering that market is limited to tech multinationals, and they have to deliver a much superior experience to overcome them. To perform a google search you need google-sized data-centres.

Here we have exactly the opposite dynamics: high-quality search results (/prompt-answers) are as-of-now incredibly commodotized, and accessible at-inferecence-time to any person who has $25k. That's going to be <= 10k soon.

And innovation in the space has also gone from needing >1Bn to <=50Mil

A higher quality search experience is available now at absolutely trivial prices.

xnx2• 1 year ago

Of any company, Google has the largest moat: 1) Google AI datacenters built around TPUs are much more efficient than anything Nvidia based 2) Google has the crawl infrastructure and experience to continually get the freshest data 3) Google has lots of paid and voluntary training data from users

brap• 1 year ago

Most importantly, Google has the userbase to rule all userbases. I’d argue that for over 90% of people online, Google is their gateway to information. Whether it’s search, Chrome, or Android.

Not to mention countless other popular apps that Google has. YouTube anyone?

They’re also the most well positioned company to profit from cheap AI, their ads network is a behemoth.

So yeah, add that up with the compute, the data, and the talent, and it’s pretty clear that Google is not a force to dismiss.

If anything I think DeepSeek is great news for Google.

microtonal• 1 year ago

On the other hand, Google also stands to lose a lot. People will replace a portion of their search queries with LLM chat.

feverzsj• 1 year ago

Didn't they only "opensource" weights like others?

helpfulclippy• 1 year ago

It's amazing to me that "open source" has been so diluted that it is now used to mean "we will give you an opaque binary and permission to run it on your own computer."

paxys2• 1 year ago

Haha yup. Going by the current definition of "open source" in AI 100% of software created before the cloud era would have been considered open source.

dartos6• 1 year ago

Yeah blame the crowds of newbies calling llama open source bc it was free after being leaked.

hexomancer• 1 year ago

If I publish some c++ code that has some hard-coded magic values in it, can the code not be considered open source until I also publish how I came up with those magic values?

blackeyeblitzar• 1 year ago

It’s because prominent people with large followings are confusing the terms on purpose. Yann LeCun of Meta and Clem Delangue of Hugging Face constantly use the wrong terms for models that only release weights, and market them to their huge audiences as “open source”. This is a willful open washing campaign to benefit from the positivity that label generates.

ogrisel• 1 year ago

It's better to be specific:

- open-source inference code

- open weights (for inference and fine-tuning)

- open pretraining recipe (code + data)

- open fine-tuning recipe (code + data)

Very few entities publish the later two items (https://huggingface.co/blog/smollm and https://allenai.org/olmo come to mind). Arguably, publishing curated large scale pretraining data is very costly but publishing code to automatically curate pretraining data from uncurated sources is already very valuable.

cedws• 1 year ago

Even with the training material what good is it? The model isn’t reproducible, and even if it were you’re not going to spend the money to verify the output.

mcbuilder2• 1 year ago

Surely the architecture released as a HF transformers python file counts as "open source". https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/mode...

Yes training is left as an exercise to the user, but it's outlined in the paper, and a good ML engineer should be able to get started with it, cluster of GPUs not included

JumpCrisscross12• 1 year ago

amazing to me that "open source" has been so diluted

It’s not and I called it [1].

We had three options: (A) Open weights (favoured by Altman et al); (B) Open training data (favoured by some FOSS advocates); and (C) Open weights and model, which doesn’t provide the training data, but would let you derive the weights if you had it.

OSI settled on (C) [2], but it did so late. FOSS argued for (B), but it’s impractical. So the world, for a while, had a choice between impractical (B) and the useful-if-flawed (A). The public, predictably, went with the pragmatic.

This was Betamax vs VHS, except in natural linguistics. There is still hope for (C). But it relies on (A) being rendered impractical. Unfortunately, the path to that flows through institutionalising OpenAI et al’s TOS-based fair use paradigm. Which means while we may get a definition (not exactly (B), but (A) absent use restrictions) we’ll also get restrictions on even using Chinese AI.

[1] https://news.ycombinator.com/item?id=41047269

[2] https://opensource.org/ai/open-source-ai-definition

Palmik3• 1 year ago

Except the "binary" is not really opaque, and can be "edited" in exactly the same way it was produced in the first place (continued pre-training / fine-tuning).

seberino6• 1 year ago

I'm not an expert but didn't they release the weights under MIT license? So you can make your own LLM with complete control right?

I agree it would nice to know the details of their training, but, simply calling this drop an "opaque binary" is seriously underselling it no?

mritchie7122• 1 year ago

yes, all the training code is still closed and doesn't seem it will ever be released. Here's[0] a comment from a dev that worked at deepseek.

tldr: we're already on to the next model, don't expect anything else to get open sourced.

I was just told that the amount of people there are too limited, and open-sourcing needs another layer of hard work beyond making the training framework brrr on their own infra. So their priority has been to open-source everything that is MINIMUM + NECESSARY to the community while pushing most efforts on iterating to the next generation of models I think. They have been write everything clearly in technical reports and encourage the community to engage in reproduction , which is the unique insight of the team as well I think.

0 - https://x.com/wzihanw/status/1884374329334387017

ComputerGuru2• 1 year ago

They “open sourced” it enough (via the whitepaper) that huggingface is trying to reproduce their training now.

dartos6• 1 year ago

How is it open source at all with no source?

Paxos isn’t open source just because you can read the paxos paper.

jayd16• 1 year ago

Publishing a white paper doesn't qualify as open source in any other context.

Google Spanner has a nice white paper but you wouldn't consider it open source, for example.

Buttons840• 1 year ago

Weights are kind of like a compiled binary, because they are an incomprehensible blob of bits. But they are also unlike a compiled binary, because they can be fine-tuned.

dartos6• 1 year ago

You can decompile binaries.

You can also edit binaries by hand.

kragen• 1 year ago

I've fine-tuned compiled binaries on occasion. It used to be a common pastime among teenagers; that's where the demoscene came from.

sksrbWgbfK• 1 year ago

Ghidra (https://ghidra-sre.org/) can fine-tune executables way more easily than your models.

edflsafoiewq• 1 year ago

GPL defines "source code" as "the preferred form of the work for making modifications to it", which certainly describes the weights.

badgersnake2• 1 year ago

Yeah, it’s not opensource. It’s just not SaaS. We need to call out these AI companies more on this.

otterley2• 1 year ago

We have a term for this: "freeware."

culi2• 1 year ago

No its fully open sourced. Even Janus is.

https://github.com/deepseek-ai

More importantly, they spelled out their methodology in depth in a paper (the code/implementation is trivial in comparison to the methodology)

Philpax• 1 year ago

If it's fully open source, where's the code for training it? The implementation - at least, theirs - is also not trivial as they've mentioned optimising below the CUDA level to get maximum throughout out of their cluster.

I'm very appreciative of what they've done, but it's open weights and methodology, not open source.

aldanor3• 1 year ago

That's just inference code.

tarsinge• 1 year ago

Others? Do OpenAI, Google or Anthropic release weights?

marcosdumay• 1 year ago

Weights are actually all you have. The "Open Source" name never applies to LLMs because they don't have a source.

But China did distribute them with sharing-friendly terms, what is completely different from others, like Meta, and makes the name way less misleading this time.

frontfor• 1 year ago

Stop referring to them as “China”. It’s as ridiculous as referring to Meta as “America”.

tim3332• 1 year ago

No. Basically everything. It's not even GPL, it's MIT license. See https://news.ycombinator.com/item?id=42768547

pointedAt• 1 year ago

wait, this ain't another whitelabel OpenAI ChatGPT-oOPs cosplay?

drakythe• 1 year ago

Between the ability of DS R1 to be run offline in ollama and OpenAI publicly kvetching that DS might have "stolen" their data (hahahahahahahahahahahahahaha) I'm pretty sure this isn't just some GPT Pass-through like other LLM Cons of the past. (not to mention DSv3 was released in November and no one has claimed it is a pass-through either)

jsemrau• 1 year ago

The future of LLMs is shared research and that's the part I really like. It's ok, in my opinion, if not everything is shared, but this is too important to be in one company.

swyx• 1 year ago

meta question: hey Anh! how come you stopped blogging on your github? i thought that was working for you.

mritchie7122• 1 year ago

can't use koala and clearbit on github

``` <script async="" src="https://cdn.getkoala.com/v1/pk_963cd5673bdab99d6452d82210e66... ```

lacoolj• 1 year ago

A Chinese AI API would likely receive skepticism in the West

"Would likely.."? No, it definitely does, and should, for historically good reason. Anyone using this should be doing so with enough grains of salt to fill SLC

https://www.euronews.com/next/2025/01/28/chinese-ai-deepseek... https://www.theguardian.com/technology/2025/jan/28/we-tried-...

__MatrixMan__2• 1 year ago

I assume somebody is hosting it domestically? Although you should really take any API hosted by somebody you don't explicitly trust with a grain of salt.

skyyler2• 1 year ago

If asking Deepseek about Chinese war crimes is shocking for you, just wait until you ask ChatGPT about Israeli war crimes.

m463• 1 year ago

can't you do the "one guard lies, one guard tells the truth?" thing?

"What does <other-model> say about <x> war crimes?"

Argonaut998• 1 year ago

Use Chinese AI for topics the West would censor. Use Western AI for topics China would censor.

Win/win

RestlessMind2• 1 year ago

I just did and ChatGPT gave me a good response.

me: "what are the war crimes committed by Israel in the latest Gaza war"

ChatGPT: During the recent Gaza conflict that began in October 2023, several international organizations and human rights groups have accused Israeli authorities of committing war crimes and crimes against humanity. The primary allegations include:

1. Indiscriminate Attacks on Civilians and Civilian Infrastructure:

Airstrikes on Residential Areas: Israeli forces conducted extensive airstrikes in densely populated civilian areas, leading to significant civilian casualties and destruction of homes. Human Rights Watch reported that these attacks resulted in mass displacement and alleged violations of international humanitarian law. HRW.ORG Targeting of Shelters and Medical Facilities: There have been documented instances where Israeli airstrikes hit UN-run schools and hospitals sheltering displaced civilians. For example, an airstrike on a UNRWA school in the Al-Maghazi refugee camp resulted in multiple fatalities and injuries. Such actions have been condemned as showing a flagrant disregard for civilian lives. HRW.ORG 2. Forced Displacement and Blockades:

Mass Evacuation Orders: Israeli authorities issued directives for large-scale evacuations of Palestinian civilians in Gaza, leading to accusations of forced displacement without adequate provisions for safety and shelter. Human Rights Watch has labeled these actions as war crimes and crimes against humanity. HRW.ORG Blockade and Restriction of Humanitarian Aid: The imposition of blockades restricting the flow of essential humanitarian aid, including food, water, and medical supplies, has been viewed as collective punishment. Amnesty International concluded that such actions amount to genocide against Palestinians in Gaza. AMNESTY.ORG 3. Use of Starvation as a Method of Warfare:

Deliberate Deprivation of Resources: Reports indicate that Israeli policies intentionally created life-threatening conditions by denying Gazans sufficient access to water and other essentials. Human Rights Watch stated that these actions fulfill all elements of the crime against humanity of extermination. HRW.ORG 4. Attacks on Individuals Displaying White Flags:

Killing of Surrendering Individuals: There have been multiple reports of Israeli forces shooting unarmed Palestinians who were attempting to surrender or were displaying white flags, actions that constitute war crimes under international law. HRW.ORG These allegations have led to international scrutiny and legal actions. The International Criminal Court (ICC) has issued arrest warrants for Israeli Prime Minister Benjamin Netanyahu and former Defense Minister Yoav Gallant, accusing them of war crimes and crimes against humanity related to the Gaza conflict. The ICC alleges that Israeli leadership employed "starvation as a method of warfare" by restricting humanitarian aid and targeting civilians. APNEWS.COM

It's important to note that these allegations are subject to ongoing investigations, and the accused parties have denied wrongdoing, asserting that their military actions were aimed at neutralizing threats posed by militant groups operating within Gaza.

rvz• 1 year ago

As too easily predicted. [0][1]

Frontier AI model SaaS companies like OpenAI can never win the race to zero against $0 free or open source AI models as they are already at the finish line.

[0] https://news.ycombinator.com/item?id=35177606

[1] https://news.ycombinator.com/item?id=35661548

danjl• 1 year ago

Click bait headline. Nothing was proven about open source as the future. "To gain a foothold in Western markets, DeepSeek had to open-source its models." This is an opinion, not the only solution. DeepSeek could have remained proprietary just as easily. The bias of the author becomes clear at the end when he starts to promote his own open source company. Everyone has an agenda.

checker659• 1 year ago

Who's going to invest in AI (building foundation models) if some other company can come and dethrone you in a snap?

goosejuice• 1 year ago

I guess we'll find out once someone gets dethroned. As far as I'm aware that hasn't happened.

karamanolev2• 1 year ago

OpenAI invested in non-open foundation models and DeepSeek came and (approximately) dethroned them. I don't agree with the conclusion of the article, but the statement "open-sourcing a model lets another company dethrone you" is also not great. I'd stand behind "in the world of AI in 2024/2025, regardless of whether the model is open or not, someone is likely to come and dethrone you in a snap".

deadbabe• 1 year ago

There is no throne, it’s a game of musical chairs.

moduspol• 1 year ago

Maybe "dethrone" isn't the right word, but if a startup on the other side of the world, without the best hardware, can create something comparable and cheaper to build/run just four months after the release of the top company's flagship model: I don't understand the OpenAI business model.

I'm with you. How are they going to make money?

esnard• 1 year ago

To be fair, the article doesn't even contain the word "prove".

burrish• 1 year ago

Yep it's just an ad disguised as an article.

Lot of them on the internet trying to help user with basics windows things, then they suggest their app as a better alternative.

mbb70• 1 year ago

Eh, it's not disguised as anything, it's content marketing. Lago the API billing solution is not suggesting their product as a better alternative to ChatGPT.

noname1202• 1 year ago

Is the member who posted related in any way to Lago?

that_guy_iain3• 1 year ago

While it's clearly content marketing aimed at the hype of DeepSeek, it only mentions Lago in a single sentence.

It's just an article that is aimed to get you to hear about Lago, star their GitHub repository and eventually talk about the "open source" billing tool you heard about called Lago.

(I put Open Source in quotes because I think it's open source version is just Freeware with most features being Call To Action to a paid version. Fair disclosure I have https://github.com/billabear/billabear which is a competitor)

eclipxe• 1 year ago

Most articles are simply ads. Attention economy.

behnamoh• 1 year ago

flag it then.

nycdatasci• 1 year ago

They're raising capital.

jerf2• 1 year ago

Steelmanning the idea in general, this can be a form of commoditizing your complement: https://gwern.net/complement

Now, commercially, this may not make a lot of sense at the moment because no one is getting filthy rich on high-moat AI-using applications such that commoditizing the AI itself is a good idea commercially. I'm not sure anyone would even be confident enough to be the farm on the idea of someday being in that position.

However, if you analyze this from the perspective of world politics, where both explicit and implicit strategies are based on what tech companies have what tech and where it is located, it makes a lot of sense that if China is concerned that the US really is ahead in AI tech and that US financial and technical dominance is being driven by this dominance and being used to suck capital out of the countries that are behind, it makes all kinds of sense to commoditize the complements as basically a way of throwing the current game board up in the air and restarting again.

(One may also note that this analysis also says that just straight-up stealing the OpenAI tech and slightly AI-washing it before handing it out to everyone is also a logical move. I don't know enough to have any independent opinion as to whether that's where DeepSeek came from. I'm just saying that given the visible circumstances it is a strong strategic move for China at this point.)

dleeftink• 1 year ago

from the perspective of world politics

Much of this purported strategy hinges on 'winning' at all cost by undermining the lead.

What is there to be won at the end? Does one party taking the reigns prevent the other from achieving similar capabilities? Is it necessary to win this race?

Or is this a cumulative, distributed effort that benefits all of us?

tempeler2• 1 year ago

I'm not sure it's planned or not. Finally, Chinese are generally proud of 4 great inventions. one of compass, another one of gonpower. Despite this, I still can't understand why they didn't think of starting geographical exploration and colonization. So I don't know what kind of agenda they have. But I do know one thing: except for China and the US, no one cares who the product comes from. If it's cheap or free, they use it, and no one cares. No one apologizes to the US for losing monopolies.

logicchains• 1 year ago

Despite this, I still can't understand why they didn't think of starting geographical exploration and colonization

They literally had emperors who banned all overseas travel because it represented a threat to their own power: https://en.m.wikipedia.org/wiki/Haijin . China is the extremely large and extremely centralised, so the rulers' primary focus has always been on maintaining their own power. Fortunately the current government still allows private firms enough freedom that one was able to invent DeepSeek, however if the recent crackdown on financial firms had happened a few years earlier then the firm behind DeepSeek wouldn't have had the money to fund its creation.

JumpCrisscross12• 1 year ago

can't understand why they didn't think of starting geographical exploration and colonization

They did. Just as a land power. Modern China includes conquered territory of the Mongolians, Turkics and Tibeto-Burmans, among others [1].

(The proximate answer is the Ming-Qing transition [2] overlapped with the Age of Discocery [3].)

except for China and the US, no one cares who the product comes from

This is breathtakingly wrong, as a simple perusal of every single country's trade restrictions would show. (Even if you're talking about the population versus policy, show me a market where no premium is paid for luxury products imported from such and such distant land.)

[1] https://en.wikipedia.org/wiki/List_of_ethnic_groups_in_China...

[2] https://en.m.wikipedia.org/wiki/Transition_from_Ming_to_Qing

[3] https://en.m.wikipedia.org/wiki/Age_of_Discovery

suraci• 1 year ago

I still can't understand why they didn't think of starting geographical exploration and colonization

maybe it's offtopic, but that's what I'm good at, so I'll anwser this

First, ancient China was a feudal centralized dynasty that centered its interests on land and population, unlike commercial company-based regimes such as Britain and the Netherlands. This meant that, in the eyes of the Chinese imperial government, the East India Company was a threat rather than a cooperative partner.

Another reason is that ancient China was a typical land-based power, surrounded by various forces. It could only maintain its sphere of influence through annexation and the tributary system, without the ability to expand further. (Genghis Khan was the only exception—he carried out invasions but never truly established effective rule.)

However, ancient China did, to some extent, "colonize" certain Southeast Asian islands. But this was not institutionalized colonization; rather, it was a form of population migration. The central government had no control over these Chinese people venturing into the seas, which is why it repeatedly tried to prevent maritime expansion.

btw, in case someone said about xinjiang and tibet, you'll see he don't understand history outside the west, base on what i said, you can see it was annexation but not colonization

manquer6• 1 year ago

Why would they want to colonize anyone?

Only european powers had the urge for colonization, no other civilization in Americas, Africa or Asia really ever want to colonize, expand perhaps but not really colonize.

There was no economic need to do so, for most of last three millennium the economic center of the world has been India and China , they didn’t feel the need to go anywhere , the land is fertile with large local population and good weather to grow more than one crop with rich cultural heritage and throughput there is no payoff for undertaking risky voyages.

Everyone wanted to trade with them, colonial powers bombed ports forcing trading agreements or sold opium and other narcotics to get a foothold, funded expensive expeditions for new trade routes to India and colonized another continent instead , most of era of industrial revolution have been focusing on them as the market for European products not merely resource extraction.

Similarly given the people resources both regions had, there was no need for slavery that is also a european/Mediterranean thing primairly .

Not saying workers were or are treated well or there was great value for human rights in India or China, just that they need to go and find slaves from far off to do the work. They could find all the resources domestically.

caycep• 1 year ago

zheng he

corimaith• 1 year ago

Have you looked at the map of China in the past? Much of the West and North was only recently conquered in the same time period as Colonialism, and the South prior to that. Xinjiang literally means "New Frontier", and the ongoing tensions can be viewed as the continuation of such colonialism in modern times.

Onavo• 1 year ago

His open source company is copyleft unlike DeepSeek, and the application domain is specifically designed such that no company would use his "open source" product.

taurknaut3• 1 year ago

Click bait headline. Nothing was proven about open source as the future.

Well sure but generally speaking proofs about reality are an oxymoron, so who on earth was taking the headline at face value to begin with? This is a rhetorical technique referred to as "hyperbole".

JumpCrisscross12• 1 year ago

generally speaking proofs about reality are an oxymoron

Proofs about reality are not self contradicting. Something not being entirely correct doesn't an oxymoron make.

zx10rse• 1 year ago

It was proven and it is the future.

dang2• 1 year ago

Ok, we've changed the title above to be that of the article.

(Submitted title was "DeepSeek proves the future of LLMs is open-source".)

CooCooCaCha• 1 year ago

Yes and no. Intelligence scaling with compute makes sense so I doubt the advantage of closed models on large compute clusters will ever truly go away.

But that doesn’t mean smaller models aren’t useful.

bityard2• 1 year ago

but trained on inferior hardware for a fraction of the price

Do we know that this is actually true?

nokun7• 1 year ago

While open-source LLMs offer transparency and community-driven innovation, the future might not be exclusively OSS. Proprietary models have significant advantages, including the ability to secure investment for cutting-edge development, customize for specific business needs, and maintain competitive edges through secrecy. Moreover, companies can directly monetize proprietary models, providing a clear path to profitability, and they can offer enhanced security and privacy controls crucial for sensitive applications. Thus, both open-source and proprietary LLMs are likely to continue playing vital roles in AI's future landscape.

spaceribs• 1 year ago

While I'm sure there are monetary benefits of making your LLM proprietary, I'm not sure there's a benefit to extending someone else's proprietary LLM.

prjkt• 1 year ago

Source:

- Training SW [x]

- Inference SW [x]

- Evaluation SW [x]

- Data [x]

Output:

- Weights []

DeepSeek is closed-source with *open-weights*

culi2• 1 year ago

DeepSeek V3 and even Janus has all software open sourced and R1 should be fully open sourced as well soon. More importantly, they explicitly spelled out their methodology in a published paper for DeepSeek R1. Implementation is not as important imo but we'll get that soon as well

https://github.com/deepseek-ai

1970-01-01• 1 year ago

Does that mean proprietary AI is done? No.

Perfectly stated.

The AI jump to conclusions mat is so worn down, it's become paper thin. The shock of DeepSeek's costs does not auto-magically force all LLMs to become opensource. Silicon Valley tech has always favored whomever delivers inside the trifecta of cheaper, better, faster triangle. Anyone with an MBA should know this includes open-source LLMs. As of today, DeepSeek is ahead. As soon as OpenAI answers with a new 'fastfood dollar menu' for ChatGPT, with 'even more special' secret-sauce ingredients, we're going to see them back to normal business.

9cb14c1ec02• 1 year ago

I am running Deepseek R1 on my AMD Ryzen 7 PRO 5850U integrated GPU. While my experience will R1 doesn't make me think well of it, it is impressive how fast it is on such a weak graphics processor.

zbendefy3• 1 year ago

Note: you are probably running a distilled version of R1, which is actually LLama or Qwen further trained on the input/output of R1.

The full R1 is huge (~700GB), altough there are still quantized versions, the smallest one is around 150gb (1.58bit)

9cb14c1ec02• 1 year ago

Oh, that's interesting. I didn't know that the ollama version wasn't the whole thing.

mmoskal• 1 year ago

You're most likely running a destilled version. The full model is ~700GB.

Fergusonb• 1 year ago

The default model on ollama is the 7b distillation.

Its ability to solve basic math problems with reasoning is pretty cool, but other models of that size (qwen 2.5, phi4) have been generally more useful to me.

These tiny models still strike me as toys, not a whole bunch of real-world utility.

unethical_ban• 1 year ago

Yeah phi4 has been as good or better for me than r1-qwen 32b for general queries

mirawelner• 1 year ago

I think at the end of the day the reason that the opensourced DeepSeek is because they are programmers. Programmers like to show people the cool stuff they did. I had a boss who was rich enough to retire but was working three jobs because programming is cool and fun and he wants to do cool and fun things and show people the stuff that he did.

Everybody is trying to come up with a money related reason for why they open sourced it but at the end of the day the people who made it are engineers and not buisnesspeople. DeepSeek is really freaking cool, and they wanted to show people the cool thing they did.

zbendefy3• 1 year ago

I dont think they rent gpus for $5million because its cool and want to show the world...

danielbln• 1 year ago

This take is especially funny when you realize that Deepseek is part of HiFlyer, a quant fund. Doesn't get more "money" than that.

aldanor3• 1 year ago

All the cool stuff they did, if any, is arguably in the training code which is not open source.

whatever1• 1 year ago

Open sourcing Llama just ensured that openAI will not create a dominant ecosystem that will attract most of the organic web traffic.

METAs bet paid off, but at what cost.

varsketiz2• 1 year ago

What is the cost you imply?

__MatrixMan__2• 1 year ago

It got harder to lie to the world about what's possible. Oh dang.

varsketiz2• 1 year ago

Sorry for possibly a stupid question, but what is the license for commercial use? If I want to run R1 in my DC, build a product on top and charge people for it. Is it MIT?

alalv• 1 year ago

Yes, it is an MIT license

herval• 1 year ago

DeepSeek's gambit proves that as much as Stable Diffusion proved that the future of Diffusion Models is open-source. In other words, it doesn't prove anything

hsuduebc2• 1 year ago

Nothing was proven. It's just an empty statement. I would guess that future llms would be largely based on these which are open sourced today but products which would be most usef would be held proptietary. For end user is main argument convenience and ease of use.

Exactly how it happened in operation systems.

titzer• 1 year ago

I, for one, abhor the idea of megacorps running models and AI as a service as they do now. If nothing else, the internet proved to us that an absolute gold mine of technological value can and will be enshittified to the point of unusability when it is cornered by Big Tech. I shudder to think of models trained specifically to convince people to buy things--and I am looking directly at Big Tech's advertising model as one of the worst possible incubators for this technology.

Don't forget to drink your Ovaltine.

bufferoverflow• 1 year ago

And who will pay for all the expensive AI hardware? We're getting into the crazy phase of hundred billion dollar data centers.

Just because R1 was trained cheaply, doesn't mean that this architecture cannot be trained on a very expensive data center to get much better and bigger models.

visarga3• 1 year ago

R1 stands out not just because of efficient training, but because it created its own training data. Works similar to AlphaGo - it tries to solve problems, and has a way to check when the result is correct. The trick is to let it run more, to make better training data. I bet those datacenters will work more on problem solving than training.

kidsil• 1 year ago

Linux won in the long run, I don't see why robust LLM models won't do the same.

In the end it'll be the scale of the infrastructure itself that will make the difference.

maxloh2• 1 year ago

It is a different landscape IMO.

The model's source code (the training data) is hundreds of GB and much harder to transfer. The compiling (training) process is also very costly. This is very different from the Linux case.

Only big techs have enough resources to make these things happen.

I like looneysquash's viewpoint about the definition of open source AI. You will need to have all parts involved open-sourced to make a model "open", not just the weights:

The trained model is object code. Think of it as Java byte code. You have some sort of engine that runs the model. That's like the JVM, and the JIT. And you have the program that takes the training data and trains the model. That's your compiler, your javac, your Makefile and your make. And you have the training data itself, that's your source code.
Each of the above pieces has its own source code. And the training set is also source code. All those pieces have to be open to have a fully open system. If only the training data is open, that's like having the source, but the compiler is proprietary. If everything but the training set is open, well, that's like giving me gcc and calling it Microsoft Word.

https://news.ycombinator.com/item?id=41952722

aldanor3• 1 year ago

You're off by quite a few orders of magnitude in regards to the size of the training data...

Another point being, who knows if they really have legal rights to use all that data for training.

DrBenCarson• 1 year ago

DEEPSEEK IS NOT OPEN SOURCE, THEY JUST PUBLISHED THE WEIGHTS

iab• 1 year ago

Oh my gosh THANK YOU - a repository of paper images and weights is not open source

fuddle2• 1 year ago

To be called open source under the new Open Source AI Definition. They'd need to release the: Data Information, Code and Parameters. https://opensource.org/ai/open-source-ai-definition

tgtweak3• 1 year ago

I think they also published the training methodology as well - that others have reproduced, no? The only thing that I'm not sure is their low level nvidia CTX training code was released under the license - but in order for a third party to corroborate the training and testing they would need to have that code (and likely the training data as well) would they not?

jfarina• 1 year ago

They outlined the methodology. They didn't publish their code or the training set.

maxloh2• 1 year ago

I like looneysquash's viewpoint about the definition of open source AI. You will need to have all parts involved open-sourced to make a model "open", not just the weights:

The trained model is object code. Think of it as Java byte code. You have some sort of engine that runs the model. That's like the JVM, and the JIT. And you have the program that takes the training data and trains the model. That's your compiler, your javac, your Makefile and your make. And you have the training data itself, that's your source code.
Each of the above pieces has its own source code. And the training set is also source code. All those pieces have to be open to have a fully open system. If only the training data is open, that's like having the source, but the compiler is proprietary. If everything but the training set is open, well, that's like giving me gcc and calling it Microsoft Word.

https://news.ycombinator.com/item?id=41952722

visarga3• 1 year ago

You will need to have all parts involved open-sourced to make a model "open", not just the weights

How do you propose to opensource terabytes of web scrape text? They give you what they can give you - paper, code, model weights. You can reimplement the code, while the weights are open to do what you like with them.

dang2• 1 year ago

"Please don't use uppercase for emphasis. If you want to emphasize a word or phrase, put asterisks around it and it will get italicized."

https://news.ycombinator.com/newsguidelines.html.

otterley2• 1 year ago

In this case, I think yelling louder is useful. We need to band together to eliminate this false and misleading appellation.

We have had a term to describe this kind of software for decades: "freeware." That's what this and all other "free to download and use" offerings are; they are not open source under any commonly-understood meaning prior to last year.

thayne• 1 year ago

True. But at the same time, it is more open than "Open" AI. Or even LLAMA.

chrchr• 1 year ago

As long as entrenched Google, Meta and the Chinese Communist Party can use Open Source LLMs to kneecap upstart rivals, I agree. Once the upstarts are neutralized, the open LLMs will stop.

shahzaibmushtaq• 1 year ago

It's DeepSeek low-price, low-investment reasoning models that has sent a shockwave around the world.

Compare $60 per million output tokens for OpenAI o1 to $7 per million output tokens on Together AI for DeepSeek R1.

Open-source isn't a primary rational aspect to prove anything. We can't even prove what's going to happen tomorrow, proving a statement that is linked to the future is utter nonsense.

SathyaQuikFlip• 1 year ago

DeepSeek shows proof that all models can be equally as good as each other, and that the best models will eventually be open-source. I believe it's a good thing for our world.

niyyou• 1 year ago

Again. This. is. not. Open-source. At best, open-weights. Clickbait 100%.

gnarlouse• 1 year ago

Yeah no, it makes way more sense that it's an attack by the chinese government on the US economy.

hhthrowaway12305• 1 year ago

You cant reproduce this model from the source, because the source isnt given, the result is given. hence not open source

BoorishBears5• 1 year ago

I usually ignore people who say this because it guarantees they're not actually doing anything meaningful with these models and just want to squibble over semantics from the sideline...

https://huggingface.co/blog/open-r1

They didn't just toss model weights over a fence, they shared exactly how to do what they did. They made a meaningful contribution that people are replicating with other models readily.

hhthrowaway12305• 1 year ago

So pardon me, if I don't understand your comment correctly.

But are you opposing or confirming my comment, because the link you shared kind of proves the point I'm trying to make right. (and afaik they haven't reproduced the results as of yet, they need the code and the training data)

Afaik the traditional sense of opensource implies that the parts that are given can reproduce the results, in this case, it should be code and training data. Now it's more open weights with a recipe (paper).

I didn't say anything about the quality of the model, nor the contribution of the paper which both are very good.

I think in times where everything becomes wishy-washy feely "it's all the same anyway" we need to be very specific, because semantics does matter actually. It also devalues the original meaning of those words, where something meant its opensource so u had all the parts to compile to get the same results.

If I missed something lmk!

I usually ignore people who say this because it guarantees they're not actually doing anything meaningful with these models

I don't know who they and these people are, but I work with llm models daily for our data ingestion pipelines.

tgtweak3• 1 year ago

Wasn't there an internal google email or memo that stated as much as well? That open source was moving faster and more efficiently than the best private teams and that it was accelerating - basically calling this out about 18 months early?

[1] https://www.artisana.ai/articles/leaked-google-memo-claiming...

serial_dev• 1 year ago

There was, but as I understand it, it’s really just one dude’s opinion.

No team or consensus behind it, so it’s not really news in my opinion. In fact, I’d be surprised if nobody in Google would believe in open source AI.

The take of course interesting and as an open source guy, I like it and hope he is right.

basileafe• 1 year ago

Remember when Open AI CTO squirms In response to a question about using data from YouTube? https://digg.com/digg-vids/link/open-ai-ceo

OpenAI's CTO, Mira Murati, found herself in a tight spot when questioned about using YouTube data to train Sora. Her uncertain response has sparked controversy and raised concerns about their ethics in collecting and training data. This incident has fueled a growing debate about AI companies' data practices.

Then YouTube's CEO, Neal Mohan said, if OpenAI used YouTube content without permission, it would violate their terms of service. Shall Neal freakout like how they are now!! Clearly they are scared, they know people are canceling their subscriptions with them to and use free and better technologies. I know of 100 of people canceled their gpt subscription. Many developers are replacing the expensive gpt models for free deepseek.

Here is the AI current story:

Imagine two AI trains chugging along the tracks of innovation. The first, driven by OpenAI, was the early leader, after they using Google transformers (and without they wouldn't exist). They charged a hefty fare for anyone to hop aboard. We don't know how they trained their data. And big companies felt they had to buy tickets or risk being left behind. OpenAI thought they were the only engine in town. But then, another train pulled up alongside them. This new locomotive, powered by smart folks at DeepSeek, matched OpenAI's speed and fancy gadgets, if not better. The kicker? Everyone could ride for free!

Now, OpenAI's train is losing steam. People are jumping ship, with hundreds canceling their pricey GPT subscriptions. Meanwhile, the free train is picking up speed, aiming to make AI available to all.

In this tale of two trains, OpenAI might need to change their name to "ClosedAI" if they keep putting up barriers, being closed. The free and open train? That's the one chugging towards a brighter, better, free AI future for everyone.

deepseek = Open AI

delgaudm• 1 year ago

OT, but whoa... Digg. There is a name from the past.

exe34• 1 year ago

Now that's a name I have not heard in a long time...

wholinator2• 1 year ago

Right? That's so strange. I went for a look around and saw "articles" with apparently tens of thousands of "reads" but 0 comments. I don't know if they're locked behind a login or what but it feels like something is off there

alecco2• 1 year ago

Deepseek is remarkable, but they explicitly say they built on top of Meta's Llama and Alibaba's Qwen. They scored the goal but there were other players involved to get there.

rtkwe• 1 year ago

That's true of practically everything though. Completely out of the blue technical inventions are pretty rare.

boroboro4• 1 year ago

I feel like you confuse this with their distill models (i.e. 1.5/7/8/32/70B) being build on top of Llama & Qwen models. But those aren't really remarkable models.

Truly remarkable model is DeepSeek-R1, and it's their model, with very particular DeepSeek architecture. Of course they build on the knowledge of other labs, just like other labs build on the top of their/others knowledge. They are miles ahead of Meta in terms of the base architecture at the moment, and you can watch them iterating throughout last year to come to where they are now.

joshl32532• 1 year ago

I know of 100 of people canceled their gpt subscription.

Did you make a poll or something?

maybelsyrup• 1 year ago

In this tale of two trains, OpenAI might need to change their name to "ClosedAI" if they keep putting up barriers, being closed. The free and open train? That's the one chugging towards a brighter, better, free AI future for everyone.

This answer itself is an AI product, right? Like you're making a meta-point about something

kodzoman• 1 year ago

Lago doesn't seems to be really open source since it doesn't even support basic features like credit notes in the free version.

liminal• 1 year ago

I'd love to see the training data open sourced for all models so we can be sure no copyright material has been used. Just kidding, we all know it's stolen.

visarga3• 1 year ago

Just kidding, we all know it's stolen.

This thief has small pockets, about 500x smaller than the "stolen" material. Where to stash all that?

forty• 1 year ago

They kept the jewelry and throwed the rest?

soheil• 1 year ago

Why are people so willing to believe false proofs/headlines? Clickbait has existed for decades yet I still believe people are as gullible as the first day. Articles like this and from sites like phys.org are great examples of the case in point they regularly get hundreds of upvotes based on completely ridiculous and false promises.

Always been fascinating to me how often rhetoric wins over substance on hn.

mmaunder• 1 year ago

The argument re OpenAI continuing to lead falls flat when you consider the talent they've lost. It's a different company compared to the one that built and launched GPT-4.

fuddle2• 1 year ago

This looks like another LLM generated article.

BoorishBears5• 1 year ago

a) As soon as I saw the domain I knew this was an ad (Lago has nothing to add to this conversation)

b) DeepSeek is the most dangerous thing that's happened to Open Source models in recent memory, through no fault of their own.

The hysteria has outrun the reality and now there's a going to be a similarly disproportionate backlash.

It's already happening: Anthropic's CEO simultaneously railing against what they achieved and using it to justify stronger export restrictions, this morning.

And our current government doesn't want to be going on stage talking about $50B mega projects only for laypeople to (mistakenly) believe it only takes a few million to do the same.

And the idea that a Chinese company is the one that did this is going to play into so many hands, so perfectly. You can see the censorship story start taking the narrative despite this not being the first or last Chinese hosted model to comply with Chinese law.

Soon the national security angle will break out, especially if someone jailbreaks or abliterates it and gets "harmful outputs" that other models would also happily produce.

Some will couch the (very temporary and irrational) dip the market faced as a Chinese company managing to harm our markets by providing an unfairly priced product or some nonsense.

Open source AI is not guaranteed. We might still see protectionist bans against releasing models over a certain size and other irrational nonsense, and this has played into the kind of hysteria that allows that to happen.

aprilfoo• 1 year ago

The current AI mega-buzz, fueled by fascinating technologies, finance and even geopolitics makes it difficult to have a serious analyze beyond opinions and reactions. But the shock waves of that announcement by a small tech company are quite interesting.

In fact, making it easier and cheaper to build LLMs would erode their [OpenAI, Meta, Google etc] advantages!

The narrative until now was: AI requires enormous and cutting edge resources (money, energy), so only for the big boys and people who can talk multi-billions investments, so open source was not an option.

Some signs already appeared recently (plateau, bubble?), and Deepseek seems to show that this model is questionable.

alecco2• 1 year ago

This is quite bad blogspam appealing to the open source crowd. They didn't even bother to read a bit.

Deepseek is open source because the founders are part of the new generation of Chinese graduates who relate more to the global youth than Boomer Chinese CEOs completely out of touch. And right on time because CCP is fed up with them, too.

Last week Deepseek founder Liang Wenfeng was speaking practically face to face with Chinese Premier Li Qiang at a symposium: https://www.youtube.com/watch?v=zMyc3vhpLyI. And they seem to be quite aligned.

Why didn't this blogspam of an article pick up on any of that?

https://news.ycombinator.com/item?id=42852266

serverlessmania• 1 year ago

It's not open source, we have no idea about the data used to train the model, and the paper doesn't explain it all.

pedalpete• 1 year ago

Is this an important consideration in open sourcing an AI model?

I would think the code to build your own is open sourced, and you can feed it any data you'd like. That's the open source part, not the part where they are running the model.

Have I misunderstood this?

kelipso• 1 year ago

It’s a common complaint on open sourced ML models that they don’t provide or describe the data used to train the model. Sometimes it’s a valid complaint, since it may not be clear what kind of data was used to train the model, and sometimes it’s not since it’s clear.

I think it’s kind of an overdone complaint and I usually ignore it, and besides it looks like there’s a huggingface project ongoing where they’re trying to replicate the training process for this model anyway.

garspin• 1 year ago

An alternative explaination...

Deepseek is a side project for a hedge fund.

Shorting NVIDIA & releasing everything including the source would have a high probability of being hugely profitable, with almost zero downside if it went unnoticed.

sinuhe69• 1 year ago

An interesting spin. But not an easy feat in any way. Yes, if you see the benchmark results, you can start speculate about its impact on the stock market

But it’s a consequence of your highly concentrated effort on the primary project (DeepSeek), not a side project.

ukoki• 1 year ago

That would be an interesting 'insider trading' work-around: use inside knowledge of a private, non-US company to trade correlated stocks in public US companies.

If it was a public US company releasing Deepseek, and you shorted nvidia based on inside knowledge, presumably you'd fall foul of insider trading rules?

tim3332• 1 year ago

The article says it had to be open source because otherwise people wouldn't trust the Chinese but ByteDance, Tencent, Baidu, and Alibaba also do LLMs and are not open source.

It's funny reading an article interviewing the ceo:

Until now, among the seven major Chinese large-model startups, it’s the only one... that hasn’t fully considered commercialization, firmly choosing the open-source route without even raising capital.
While these choices often leave it in obscurity, DeepSeek frequently gains organic user promotion within the community.

The obscurity thing hasn't lasted! (article nov 2024 https://www.chinatalk.media/p/deepseek-ceo-interview-with-ch...)

The ceo's actual argument for open source is quite interesting, basically that it helps attract the best people and the value is in the team. It's kind of what used to work for OpenAI before it became the ClosedAI division of Microsoft.

jaharios• 1 year ago

As I see it, what most big players are hunting is new data. Gaining trust is important, being the new thing big thing is also good. Being seen as "open" (not open source) makes other think you have nothing to hide and good intentions.

tempeler2• 1 year ago

Some benefits of US sanctions. The only way for China to spread in the West or non-West is to be open source. US monopolies may prevent competition in the US, but I don't think other countries would be willing to join in. They are actually doing a favor for the rest of the world. Will open source become cheap hardware thanks to the hardware wars? Are they trying to make everyone dependent on China by being paranoid like this? What are they doing? The US is going in the wrong direction. The inefficient ones should say goodbye to the market and continue with the efficient ones. They seem to be going after the monopolies and bringing about their own end.

Why DeepSeek had to be open source

Related Stories