OpenAI says it has evidence DeepSeek used its model to train competitor

https://www.ft.com/content/a0dfedd1-5255-4fa9-8ccc-1fe01de87ea6

by timsuchanek • 1 year ago

747 1474 1 year ago

ft.com

udev• 1 year ago

https://archive.is/KiSYM

cratermoon5• 1 year ago

Ironic, OpenAI claiming someone else stole their work.

vinni2• 1 year ago

How would they prove they used it’s model. I would be curious to know their methodology. Also what legal actions OpenAI can take? can DeepSeek be banned in US?

iforgot2210• 1 year ago

They might show DeepSeek's model calling itself ChatGPT, which users have already alleged. Same as how Cisco proved Huawei was stealing router code.

Except in this case, nothing was stolen, unless they want to call ChatGPT's own training on source data theft too.

freehorse5• 1 year ago

ChatGPT outputs are all over the internet. It is harder to prove that deepseek used specifically o1 for training, instead of a lot of chatgpt output ending up in the training set from other sources.

paul_e_warner4• 1 year ago

If you read the article (which I know no one does anymore)

OpenAI and its partner Microsoft investigated accounts believed to be DeepSeek’s last year that were using OpenAI’s application programming interface (API) and blocked their access on suspicion of distillation that violated the terms of service, another person with direct knowledge said. These investigations were first reported by Bloomberg.

belter3• 1 year ago

The subtitle is the gold... : "White House AI tsar David Sacks raises possibility of alleged intellectual property theft"

conartist62• 1 year ago

lolololololololol

vrighter• 1 year ago

So what? They probably paid for api access just like everyone else. So it's a TOS violation at worst. Go ahead, open a civil suit in the US against an entity the US courts do not have jurisdiction over and quit whining...

jhickok• 1 year ago

open a civil suit in the US against an entity the US courts do not have jurisdiction over

Yeah, over a Chinese company no less.

ForHackernews• 1 year ago

What's good for the goose is good for the gander. Obviously a transformative work and not an intellectual property violation any more than OpenAI injesting every piece of media in existence.

dagelf• 1 year ago

Injesting is sure the right take. What a circus!

amarcheschi2• 1 year ago

I quite like a scenery where llm output can't be copyrighted, so that it is possible to eventually train a llm with data from the previous one(s)

layer87• 1 year ago

OpenAI argues it’s a violation of their terms of service. So there are legal issues if it can be proven.

mannewalis2• 1 year ago

But OpenAI's model isn't open source, how would they distill knowledge without direct access to the model?

Palmik5• 1 year ago

Legal issues for who?

Company A pays OpenAI for their API. They use the API to generate or augment a lot of data. They own the data. They post the data on the open Internet.

Company B has the habit of scraping various pages on the Internet to train its large language models, which includes the data posted by Company A. [1]

OpenAI is undoubtedly breaking many terms of service and licenses when it uses most of the open Internet to train its models. Not to mention potential copyright violations (which do not apply to AI outputs).

[1]: This is not hypothetical BTW. In the early days of LLMs, lots of large labs accidentally and not so accidentally trained on the now famous ShareGPT dataset (outputs from ChatGPT shared on the ShareGPT website).

top_sigrid• 1 year ago

https://archive.is/KiSYM

lawlessone• 1 year ago

So they're mad someone did exactly what they did?

exe343• 1 year ago

no, no, it's completely different. "open"AI stole from poor people. DeepSeek stole from a $1T company. that's illegal!

whatshisface6• 1 year ago

It's reasonably likely that a lot of people linked to the federal government want to ban DeepSeek. You can tell it's being presented away from "they gave us a free set of weights" and towards "they destroyed $1T of shareholder value." (By revealing that Microsoft et al. paid way too much to OpenAI et al. for technology that was actually easy to reinvent.)

fullshark2• 1 year ago

Would it even matter? Isn't the cat out of the bag and everything they did repeatable by an American research team?

Cumpiler692• 1 year ago

It matters because their goal was hyping up how advanced and difficult their tech is, propping up their valuations.

DeepSeek proved the emperor had no clothes and wiped out a lot of their valuation when investors saw reaching parity to Chtgpt is not really that difficult.

bhouston6• 1 year ago

It doesn't matter from the US government perspective if all of the tech is replicated by US companies and US user continue to use US AI technology. But if US users start to use Chinese AI tech, then protectionism urges will appear that will likely figure out how to ban its use or subject it to large tariffs (e.g. TikTok, BYD, network equipment, solar panels, etc.)

whatshisface6• 1 year ago

American researchers had already made enough progress to prove that LLMs were not an incomprehensible trade secret based on years of secret knowledge - investors and tech executives were simply lead to believe otherwise. Well-connected people are probably very mad about this and they may try to lash out like the emotional human beings they are.

Cumpiler692• 1 year ago

By revealing that Microsoft et al. paid way too much to OpenAI et al. for technology that was actually easy to reinvent.

That's why it's called a bubble. Pretty sure my great great grandad also overpaid for some tulips.

toomuchtodo3• 1 year ago

"they destroyed $1T of shareholder value." (By revealing that Microsoft et al. paid way too much to OpenAI et al. for technology that was actually easy to reinvent.)

The value was highly speculative, an illusion created by PR and sentiment momentum. "Hype value" not real value (unless you're able to realize it and dump those bags on someone else before fundamentals set in). Same thing happening with power companies downstream of the discovery that AI is not going to be a savior of sagging electricity demand. Overdriving the fundamentals is not value destruction, it is "I gambled and lost."

https://www.bloomberg.com/news/articles/2025-01-28/deepseek-... | https://archive.today/mCemf

"In the short run, the market is a voting machine but in the long run, it is a weighing machine."

cft2• 1 year ago

Since the time when companies en masse stopped paying cash dividends on owned shares, the value has become highly speculative. In the absence of dividend payments, the stock pricing mechanism is not essentially different from Solana or Ethereum "price" discovery.

DebtDeflation3• 1 year ago

What they really destroyed was the idea that OpenAI would be able to charge $200/month for their ChatGPT Pro subscription which includes o1. That was always ridiculous IMO. The Free tier and $20/month Plus tier along with their API business (minus any future plan to charge a ridiculous amount for API access to o1) will be fine.

scarface_748• 1 year ago

The two podcasters who do the Acquired podcast spoke to Ballmer about some of Microsoft’s failed initiatives and acquisitions. He told them that at the end of the day “it’s only money”.

All of the BigTech companies have enough cash flow from profitable lines of business to make speculative bets.

onlyrealcuzzo9• 1 year ago

Theoretically this should be good for OpenAI - in that they can reduce their costs by ~27x and pass that along to end users to get more adoption and more profit.

ceejayoz19• 1 year ago

No; those costs were their moat.

wturner• 1 year ago

Capitalism - a system where rational actors make informed decisions.

blantonl4• 1 year ago

Nah, those costs were for their doomsday bunkers and crypto purchases, and maybe a house or 3

askl• 1 year ago

They could pivot to being a wrapper around DeepSeek. That would also save a lot of R&D costs.

mirzap• 1 year ago

Training costs are not the same as inference costs. DeepSeek (or anyone hosting DS largest model) will still need a lot of money and a bunch of GPU clusters to serve the customers.

btbuildem• 1 year ago

pass that along to end users

I don't think that's at all likely in the current economic system

aprilthird20214• 1 year ago

Banning it will not bring back the value

duxup• 1 year ago

When it comes to the executive branch's role in banning something. I'm not convinced they're even honest about it / what the context even is.

Trump wanted to ban Tiktok before... and then simply chose not to / forgot about it.

Next round congress acted, and Trump delayed it and has said that he is interested in his friends buying it.

Is there really a competitive plan here or is it just fishing for payouts / grifting for allies?

The context is always about competition, but I'm not even sure that's their plan.

the_sleaze_• 1 year ago

"easy to reinvent" often comes after "hard to invent"

mromanuk• 1 year ago

At Microsoft’s size, they don’t care; they just buy out others.

IAmGraydon2• 1 year ago

Yeah they're setting this up to ban it. Crazy that they think this kind of approach will work in any way. Banning H100s didn't work, and actually pushed them to innovate. Now someone has found a more efficient way to train a model and they decide the best way forward is for the US not to benefit from access to it? This is clear evidence of collusion between OpenAI and the US Government to disadvantage competitors. Beyond that, it will never work. If they need to be reminded of just how little power they have to control the distribution of open source models, I think we would all be happy to enlighten them.

Buttons8403• 1 year ago

I saw a some Europeans hoping that the US would ban DeepSeek, because then there would be less traffic interfering with their own DeepSeek queries.

The US can ban all they want, but if the rest of the world starts preferring Chinese social media, Chinese AI, and Chinese websites in general, the US is going to lose one of its crown jewels.

The way the US behaves is a problem and makes a lot of people prefer alternatives just for the sake of avoiding the US, which is why it's important that the US get along with other nations, but--well, about that...

nozzlegear5• 1 year ago

Agreed, you've highlighted one of the key problems with protectionism and nativism. Banning competition just weakens America's global influence, it doesn't make it stronger.

karel-3d• 1 year ago

EU will ban DeepSeek sooner because of (lack of) GDPR compliance

tensor6• 1 year ago

I've recently cancelled my Github Copilot subscription and now use Mistral. When the US starts threatening allies with tariffs or invasion, using US services becomes a major business risk.

dkjaudyeqooe4• 1 year ago

Not sure I agree with your premise, but what exactly are they going to ban?

They can stop DeepSeek doing various things commercially I guess, but stopping Americans using their ideas is simply impossible and stopping use of their source or weights would be (likely successfully) challenged under the first amendment.

There is no law against simply destroying trillions of dollars of shareholder value.

thedevilslawyer• 1 year ago

Heh, not yet..

dfxm12• 1 year ago

There's a lot of egg on people's faces now. DeepSeek shows there's nothing special about America or its economic system that breeds innovation. DeepSeek shows how these tech oligarchs greatly overplayed their hand and along with the president, bamboozled the taxpayer to enrich each other. I just hope the voters remember this in 2 years, 4 years and beyond.

dtquad2• 1 year ago

DeepSeek shows how these tech oligarchs greatly overplayed their hand and along with the president, bamboozled the taxpayer to enrich each other.

How much taxpayer money has gone to OpenAI and Anthropic? They are the two big sinners in closed AI.

iforgot2210• 1 year ago

If the allegations are true, the special thing about OpenAI is that it didn't have to be trained off DeepSeek. But either way, you maybe don't want to invest billions in something if someone else will be able to copy it for less.

dtquad2• 1 year ago

It's reasonably likely that a lot of people linked to the federal government want to ban DeepSeek.

It took them years and years to move forward with the ban ok Tiktok and it still hasn't been banned yet. There is no way they are going to ban some MIT-licensed weights.

"they destroyed $1T of shareholder value."

The market has largely recovered.

tokioyoyo2• 1 year ago

There is big American money invested in TikTok. That doesn’t seem to be the case for DeepSeek.

leesec• 1 year ago

It was so easy it cost hundreds of millions of dollars, only one company has done it and they had to lie about it

nullbyte• 1 year ago

I think the real concern from the govt's perspective is data privacy, since all the chat messages are stored on Chinese servers

jasoneckert• 1 year ago

What I find the most comical about this is that the whole situation could be loosely summarized as "OpenAI is losing its job to AI."

zbshqoa3• 1 year ago

Realistically that's the actual headline. Only another AI can replace AI, pretty much like LLMs / Transformers have replaced "old" AI models in certain task (NLP, Sentiment Analysis, Translation etc) and research is in progress for other tasks as well performed by traditional models (personalization, forecasting, anomaly detection etc).

If there's a better AI, old AI will lose the job first.

troyvit2• 1 year ago

NLP, Sentiment Analysis, Translation etc

As somebody who got to work adjacent to some of these things for a long time, I've been wondering about this. Are LLMs and transformers actually better than these "old" models or is it more of an 80/20 thing where for a lot less work (on developers' behalf) LLMs can get 80% of the efficacy of these old models?

I ask because I worked for a company that had a related content engine back in 2008. It was a simple vector database with some bells and whistles. It didn't need a ton of compute, and GPUs certainly weren't what they are today, but it was pretty fast and worked pretty well too. Now it seems like you can get the same thing with a simple query but it takes a lot more coal to make it go. Is it better?

Keyframe2• 1 year ago

also, China doing in IP what it's better at and way more experienced than USA - stealing.

rchaud• 1 year ago

This kind of blithe commentary is 20 years out of date and reminiscent of 1970s criticisms of the Japanese car industry.

t43562• 1 year ago

Who says America is less good at it? Hasn't the US nicked a lot of other people's ideas at some point or other?

elzbardico3• 1 year ago

We stole a lot of scientists and equipment from Germany after WW2. That's a big part of how we get to the moon.

mattgreenrocks• 1 year ago

OpenAI should be excited that it has been freed of the tedious tasks of building AI and now they can focus on higher level and more creative things.

Sateeshm2• 1 year ago

focus on higher level and more creative things.

But that's what OpenAI's costumers were supposed to do.

pphysch5• 1 year ago

OpenAI should be, but OpenAI died a while ago

JoshTko2• 1 year ago

I wish I could upvote this twice

blantonl4• 1 year ago

Otherwise known as a race to the bottom

bwfan123• 1 year ago

ha, the story is filled with ironies.

OpenAIs $200 closed-ai uppended by hedge-funds free side-project

Quant geeks outcompete overpaid silicon valley devs etc.

Basically, hubris gets its comeuppance which is a david vs goliath biblical archetype which is why this drama grips all of us.

jeffreyq• 1 year ago

seems ironic that the turns have tabled. "silicon valley devs" were the analogous "quant geeks" underdogs that unseated the ossified incumbents.

That said, I feel like "quant geeks" aren't quite underdogs compared to silicon valley devs. wdyt?

rooroobooragool2• 1 year ago

This is really the top take in this thread. Why should OpenAI be any different than all the others they they've ripped off.

nikeee• 1 year ago

More like

"OpenAI is losing its job to open AI."

semking8• 1 year ago

This is absolutely hilarious! :)

ClosedAI scraped human content without asking and they explained why this was acceptable... but when the outputs of their training corpus is scraped, it is THEIR dataset and this is NOT acceptable!

Oh, the irony! :D

I shared a few screenshots of DeepSeek answering using ChatGPT's output in yesterday's article!

https://semking.com/deepseek-china-ai-model-breakthrough-sec...

marricks2• 1 year ago

Also, DeepSeek is allegedly... better? So saying they just copied ClosedAI isn't really sufficient of an answer. Seems to be just bluster because the US Govt would probably accept any excuse to ban it, see TikTok.

semking8• 1 year ago

I never said they are just a clone! There's an actual tech breakthrough!

Read the two following sections of my blog post:

1. "Distilled language models"

2. "DeepSeek: Less supervision"

beAbU4• 1 year ago

How can they ban something thats open source that you can just run on your own hardware?

throwup2388• 1 year ago

It’s not better. In most of my tests (C++/QT code) it just runs out of context before it can really do anything. And the output is very bad - it mashes together the header and cpp file. The reasoning output is fun to look at and occasionally useful though.

The max token output is only 8K (32K thinking tokens). O1 is 128k, which is far more useful, and it doesn’t get stuck like R1 does.

The hype around the DeepSeek release is insane and I’m starting to really doubt their numbers.

mritchie712• 1 year ago

openai should pay creators, but:

1. scraping the internet and making AI out of it

2. using the AI from #1 to create another AI

are not the same thing.

zbshqoa3• 1 year ago

Number 2 is already possible with open models. You can do distillation using Llama, which could likely be doing #1 to build their models (I'm not sure it's the case though)

bugglebeetle2• 1 year ago

Yeah, #1 is way worse and #2 falls under “turnabout is fair play.”

epse• 1 year ago

#1 destroys peoples willingness to publish and unfairly hogs bandwidth / creates costs for small hosters

#2 makes a big corp a bit angry

Indeed not the same thing

latexr• 1 year ago

are not the same thing.

You’re right. The second one is far more ethical. Especially when stealing from a thief.

Doesn’t Sam Altman keep parroting they’re developing AI “for the good of humanity”? Well then, someone taking their model and improving on it, making it open-source, having it consume less, and having a cheaper API, should make him delighted. Unless he *gasp* was full of shit the whole time. Who could have guessed?

Palmik5• 1 year ago

I agree, (2) seems much less problematic since the AI outputs are not copyrightable and since OpenAI gives up ownership of the outputs. [1]

So, if you really really care about ToS, then just never enter into a contract with OpenAI. Company A uses OpenAI to generate data and posts it on the open Internet. Company B scrapes open Internet, including the data from Company A [2].

[1]: Ownership of content. As between you and OpenAI, and to the extent permitted by applicable law, you (a) retain your ownership rights in Input and (b) own the Output. We hereby assign to you all our right, title, and interest, if any, in and to Output.

[2]: This is not hypothetical. When ChatGPT got first released, several big AI labs accidentally and not so accidentally trained on the contents of the ShareGPT website (site that was made for sharing ChatGPT outputs). ;)

sksrbWgbfK• 1 year ago

2. using the AI from #1 to create another AI

2. scraping the AI from #1 and making AI out of it

haswell• 1 year ago

Yes, they are different actions.

But arguably these actions share enough characteristics that it’s reasonable to place them in the same category. Something like: “products that exist largely/solely because of the work of other people”. The nonconsensual nature of this and the lack of compensation is what people understandably take issue with.

There is enough similarity that it evokes specific feelings about OpenAI when they suddenly find themselves on the other side of the situation.

Winsaucerer• 1 year ago

I'm genuinely not sure which one you think is worse (if any). (1) seems worse, but your reply suggests to me maybe you think (2) is worse.

jillyboel• 1 year ago

You're right, (1) is violating the rights of a large portion of the population, (2) is violating the rights of one company

tw19845• 1 year ago

#1 is stealing from all average joes ever lived on earth

#2 is taking advantages from closedAI.

they are indeed different

pilooch• 1 year ago

Any ML based service with an API is basically a dataset builder for more ML. This has been known forever and is actually a useful "law" of ML-based systems.

sho_hn14• 1 year ago

Aye, this should be obvious even to non-technical folks. Much has been written about how LLMs regurgitate the data they were trained on. So if you're looking for data to train on, you can certainly extract it there.

Plus of course for people within the tech bubble, plenty of research results on the value of synthetically augmented and expanded training data that put the impact past just regurgitating source data.

This whole episode is a failure of reporting what to expect next and projecting running costs etc. most of all.

amelius4• 1 year ago

This is why models should be open. Or at least they should have a local option.

stackghost4• 1 year ago

[flagged]

bloomingkales2• 1 year ago

I personally love this chef's kiss of a flip flop sam did here:

https://blog.samaltman.com/trump

https://www.reddit.com/r/YAPms/comments/1i7ry5m/sam_altman_g...

Only a truly talented piece of shit can be as prolific as this.

"He is irresponsible in the way dictators are."

Chef's kiss.

Edit:

Kids, don't aspire to be like Altman. We as a community need to espouse more values than tech is gonna tech.

652• 1 year ago

Yeah I don't know, Altman is a sociopath who is now trying to get intertwined with local governments (SF) as well as the federal government. He's going to do a lot of weaseling to get what he wants: laws that forcibly make OpenAI a monopoly.

Society will always have crazy sociopaths destroying things for their own gain, and now is Altman's turn.

api4• 1 year ago

I'm sure him lining up to kiss Trump's ring for some kind of bailout is not a coincidence.

blackeyeblitzar4• 1 year ago

I don’t care for Sam Altman and his general untrustworthy behavior. But DeepSeek is perhaps more untrustworthy. Models from American companies at least aren’t surprising us with government driven misinformation, and even though safety can also be censorship, the companies that make these models at least openly talk about their safety programs. DeepSeek is implementing a censorship and propaganda program without admitting it at all, and once they become good at doing it in less obvious ways, it can become very damaging and corrupt the political process of other societies, because users will trust the tools they use are neutral.

I think DeepSeek’s strategy to announce a misleading low cost (just the final training run that optimizes a base model that in turn is possibly based on OpenAI) is also purposeful. After all, High Flyer, the parent company of DeepSeek, is a hedge fund - and I bet they took out big short positions on Nvidia before their recent announcements. The Chinese government, of course, benefits from a misleading number being announced broadly, causing doubt among investors who would otherwise continue to prop up American technology startups. Not to mention the big fall in American markets as a result.

I do think there’s also a big difference between scraping the Internet for training data, which might just be fair use, and training off other LLMs or obtaining their assets in some other way. The latter feels like the kind of copying and industrial espionage that used to get China ridiculed in the 2000s and 2010s. Note that DeepSeek has never detailed their training data, even at a high level. This is true even in their previous papers, where they were very vague about the pre training process, which feels suspicious.

tootie• 1 year ago

The coup against him is looking more and more like a huge "I told you so" moment.

Dansvidania3• 1 year ago

Indeed. First thing I thought was "call a wahmbulance!".

benreesman• 1 year ago

The guy is a total fucking psycho and the rest of the board are no gems either.

Their failure is important at a minimum to the future of the United States if not the world.

dang9• 1 year ago

Ok, but please don't break HN's rules when commenting here.

You may not owe altmen better, but you owe this community better if you're participating in it.

https://news.ycombinator.com/newsguidelines.html

belter3• 1 year ago

So it is true, they run out of Data to steal? :-)

And then where DeepSeek steal from next? Do they steal from themselves? Do they steal the stolen models they stole from the stolen data?

The AI Ponzi scheme...

troyvit2• 1 year ago

Exactly this, especially as journalism melts down into slag. Soon all anybody will have to train on is social media, Wikipedia and GitHub, and that last one will slowly be metastasized by AI-generated code anyway.

It reminds me of 1984 in a sense. "Don’t you see that the whole aim of Newspeak is to narrow the range of thought? In the end we shall make thoughtcrime literally impossible, because there will be no words in which to express it."

Unlike 1984 I don't see this winnowing of new concepts as purposeful, but on the other hand I keep asking myself how we can be so stupid as to keep doing it.

TypingOutBugs• 1 year ago

Screw OpenAI, they scrape us without issues so someone scraped them. No issues with this.

coliveira11• 1 year ago

But the government will now claim this is against "national security". Only American companies are allowed to commit this kind of "sleight of hand".

pen2l• 1 year ago

While all of this is true, that DeepSeek wouldn't be here were it not for the research that preceded it notably Google's paper, then Llama, and ChatGPT which they're modeled after, its release still did something profound to their psyche, the motivation and self-actualization this instills to the Chinese. They witnessed the power of their accomplishments: a side-hustle project knocked off an easy trillion. This is only egging them on and will serve to ramp up their efforts even more.

Separately, I do think that now that the Chinese leadership saw this, that they have the chops to pull this off and then some, they are probably going to rein in future innovations; they'll likely demand that the big future discoveries remain closed-sourced (or even unannounced/unpublicized).

tedivm4• 1 year ago

OpenAI wouldn't be here without the work that Yann Lecun did at Facebook (back when it was facebook). Science is built on top of science, that's just how things work.

openrisk2• 1 year ago

they'll likely demand that the big future discoveries remain closed-sourced

Depends on whether they want these tools to be adopted in the wider world. Rightly or wrongly there is a lot of suspicion in the West and an open source approach builds trust.

hn_throwaway_996• 1 year ago

While all of this is true, that DeepSeek wouldn't be here were it not for the research that preceded it (notably Llama), and ChatGPT which they're modeled after...

If the allegation is true (we don't know yet), then what you've written perfectly proves the point everyone is making. ChatGPT wouldn't be here if it weren't for all the research and work that preceded it in terms of tons of scrapable content being available on the Internet, and it's not like OpenAI invented transformers either.

Nobody is accusing DeepSeek of hacking into OpenAI's systems and stealing their content. OpenAI is just saying they scraped them in an "unauthorized" manner. The hypocrisy is laughably striking, but sadly nobody has any shame anymore in this world it seems. Play me the world's tiniest violin for OpenAI.

stravant• 1 year ago

Yes, and what does preceding research do? Get followed by more research building on it.

dismalaf4• 1 year ago

Don't forget all the research that came before OpenAI and ChatGPT...

nicce4• 1 year ago

We wouldn't be here discussing if nobody invented internet... nor these models had training data at all.

Separately, I do think that now that the Chinese leadership saw this, that they have the chops to pull this off and then some, they are probably going to rein in future innovations; they'll likely demand that the big future discoveries remain closed-sourced (or even unannounced/unpublicized).

How do we know that this is not already happening with OpenAI/Meta and the U.S. government at some level? The concept of power is equal, whether we wanted it or not. We don't have to pretend to be "better" all the time.

coliveira11• 1 year ago

They really lost their minds. They're all scared and worried because companies in other countries can also access the same data they stole from the Internet.

rvz• 1 year ago

They have been out-grifted by DeepSeek and OpenAI is not happy about someone out-shining them on that.

The best part is "their IP" was humanity's scraped content and they are angry that DeepSeek did their job for them and gave it away for free.

cscurmudgeon6• 1 year ago

Scraping data is different from scraping outputs from a model.

jacobgorm4• 1 year ago

No it is not, data is data, whether it gets loaded from a file on disk or generated by multiplying lots of matrices.

idle_zealot2• 1 year ago

Like, in a strict literal sense, sure? Do you mean to make a claim about moral or legal differences?

openrisk2• 1 year ago

Because the data is mine and the model is yours?

wkz• 1 year ago

Technically, sure. What is the moral distinction though?

Rebelgecko• 1 year ago

Copyright is weird and often legal ≠ moral, but I'm having a hard time constructing a mental model where it's ok to scrape a novel written by a person but it's not ok to scrape a story written by chatgpt

28304283409234• 1 year ago

ClosedAI? StolenAI!

gruez3• 1 year ago

The picture at the end showing deepseek's privacy policy and being concerned that it's "a security risk" is hilarious[1]. Basically every B2C company collects this sort of information[2], and is far less intrusive than what social networks collect[3]. But because it's Chinese and at the risk of overtaking Western companies, people are suddenly worried about device information and IP addresses?

[1] https://semking.com/wp-content/uploads/2025/01/DeepSeek-1024...

[2] https://www.bestbuy.com/site/help-topics/privacy-policy/pcmc...

[3] https://www.facebook.com/privacy/policy/

semking8• 1 year ago

One of my core followers named Bruno basically said the same thing under my Linkedin post yesterday:

https://www.linkedin.com/posts/organic-growth_deepseek-the-o...

I welcome friction, so I'll be blunt: I disagree with you, not because what you are saying is wrong but because you only consider systematic data collection.

That's not the issue here.

There's a difference between democracies like the United States or European countries, no matter how IMPERFECT they are, and a dictatorship that does not allow dissenting opinions.

There's a difference in how the data collected will be used.

Freedom of speech, even when it is relative, is better than totalitarianism.

scotty796• 1 year ago

"That's hilarious!" was my first reaction as well, when I heard about it the first time. When I came to HN and saw this story on top I was hoping this was the top comment. I was not disappointed.

US AI folk were leading for two years by just throwing more and more compute at the same thing that Google threw them like a bone years ago (namely transformers). They made next to no innovation in any area other than how to connect more compute together. The idea of additional inference time compute, looping the network back on its own outputs, which is the only significant conceptual advancement of last years was something I, as a layman, came up with after few days of thinking why AI sucks and what can be done to make it able to tackle problems that require iterative reasoning. They announced it few weeks after I came up with the idea, so it was in the works for some time, but it shows you how basic idea it was. There was nothing else.

Suddenly when there comes a small company that introduced few actual algorithmic advancements which resulted in 100x optimization which is something expected with algorithmic optimizations, the big AI suddenly went into full "dog ate my homework" mode. Blaming everyone and everything around.

Let's not mention the fact that if full outputs of their models could enable them to train a better model at 1% cost then it puts them in even worse light that they didn't do it.

ryanobjc2• 1 year ago

It’s not often you get 100x optimization with some small improvements so I’m kind of skeptical.

We have and apples and oranges thing here which deepseek is intentionally leaning into. They get very cheap electricity and are bragging about their cheap cost, and OpenAI etc typically brag about how expensive their training is. But it’s all pr and lies.

Leary• 1 year ago

Does this mean when you use OpenAI as an enterprise customer, they can see exactly the queries and answers? So much for privacy!

api4• 1 year ago

So far the whole business model of Silicon Valley since social media has been to monetize other peoples' content given out for free. The whole empire is built on this.

I wonder if this is going to come to an end through a combination of social media fatigue, social media fragmentation, and open source LLMs just giving it all back to us for free. LLMs are analogous to a "JPEG for ideas" so they're just lossy compression blobs of human thought expressed through language.

barnabee2• 1 year ago

So far the whole business model of Silicon Valley since social media has been to monetize other peoples' content given out for free. The whole empire is built on this.

It cannot die soon enough

okdood64• 1 year ago

Not to mention the total dodge when Murati was asked about training on the YouTube corpus during that television interview.

Sorry for the Short: https://www.youtube.com/shorts/M0QyOp7zqcY

the_arun2• 1 year ago

I think the point is - OpenAI scraped public data - d1 - Trained their model to produce output - d2 - DeepSeek used d2 to reinforce their model

OpenAI is mad about d2 (not d1). I'm not sure using public data is "stealing". In summary, these are two different things & need to be separate.

redleader55• 1 year ago

You say "public", but what I think you mean is "publicly available". Even publicly available data has copyrights, and unless that copyright is "public domain", you need to follow some rules. Even licenses like Creative Commons, which would be the most permissive, come with caveats which OpenAI doesn't follow [0].

It is unclear if someone breaking someone else's copyright to use A can claim copyright on a work B, derived from A. My point is that OpenAI played loose with the copyright rules to build its various models, so the legality of their claims against DeepSeek might not be so strong.

[0] https://creativecommons.org/share-your-work/cclicenses/

xbar• 1 year ago

OpenAI (sc)raped all the data it could. I do not accept your assertion that d1 was "public." It was accessible, for certain.

OpenAI asserts 1. d2 was used by DeepSeek 2. All d2 belongs to OpenAI exclusively

Both are debatable for large number of reasons.

rubslopes2• 1 year ago

Our mission is to ensure that artificial general intelligence benefits all of humanity.[1]

Well, I guess they really helped make this a reality!

[1] https://openai.com/about/

didip• 1 year ago

fr fr, ClosedAI is being a comedian right now.

They scraped literally all the content of the internet without permissions. And I won't even be surprised if they scraped the output of other LLMs as well.

adzm• 1 year ago

Why does this post use DeepSink instead of DeepSeek at apparently random places? Is that just a pejorative pun like ClosedAI?

skeeter20204• 1 year ago

I share the sentiment here, but asking as a noob: does this mean the performance comparison is not really apples to apples? If it required the distillation of the expensive model in order to get such good results for a much lower price, is that shady accounting?

schmit• 1 year ago

Even more hilarious given their own charter:

We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.
Our primary fiduciary duty is to humanity. We anticipate needing to marshal substantial resources to fulfill our mission, but will always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit.
We will actively cooperate with other research and policy institutions; we seek to create a global community working together to address AGI’s global challenges.

semking8• 1 year ago

Ah yes: "duty to humanity"

radicality• 1 year ago

I liked Matt Levine’s newsletter few days ago where he hypothesized scenarios where it’s much more profitable to short your competitors, then release a much better version of some widget completely free, and then profit $$$. Which is plausible here too, considering DeepSeek is made by a hedge fund.

greasegum• 1 year ago

Came here to mention this too. Seem almost so obvious that I'm surprised this isn't the dominant angle.

freehorse5• 1 year ago

How would that work out here though? "Open"AI is not publicly traded. Any kind of shorting would be quite indirect.

Imustaskforhelp3• 1 year ago

Yes the irony is so thick in the air that it can be cut through using a swiss knife lol

I had literally come to this post to say the same. You beat me to it.

USA is going crazy over deepseek and to me , it just shows that the world is a black swan , an AI bubble.

I am not saying AI has no use. I regularly use it to create something , but its just not recommended. I am going to stop using AI , to grow my mind.

And its definitely way overpriced. People are investing so much money without seeing the returns? , and I think people are also using AI because of a sense of FOMO , I don't know , to me its funny .

I really really want to create a index fund with strictly no AI companies. Since this doesn't feel diversified enough. Like sure nvidia gave a quarter of return the last year , but I mean , at this point , it almost feels the same as that of bitcoin. The reason I don't / won't invest in bitcoin is I don't want "that" risk.

This has been a boggling year.

I have realized that the world is crazy. Truly. Trump winning from going to the point of getting shot to deepseek causing nvidia / american stock market to go down , heck even bitcoin! , its so crazy , trump launching his meme coin. If the world is crazy. Just be the sane person around. You will stick around , that's my philosophy. I won't jump on AI wandwagon . But its still absolutely wild & horror seeing how a "sideproject" (deepseek) absolutely put american stock market in shambles.

I want more diversifaction. I am not satisfied with the current system. This feels like a bubble and I want no part in it.

amelius4• 1 year ago

It looks like they want to spin this as "DeepSeek copied OpenAI". The general public/media might actually believe this is what happened.

breakitmakeit• 1 year ago

As the article points out, they are arguing in court against the new york times that publicly available data is fair game.

The questions I am keenly waiting to observe the answer to (because surely Sam's words are lies): how hard is OpenAI willing to double down on their contradictory positions? What mental gymnastics will they use? What power will back them up, how, and how far will that go?

snakeyjake• 1 year ago

When large sums of money are involved the techbros will burn everything down, go scorched earth no matter what the consequences, to keep what they believe they're entitled to.

ADeerAppeared• 1 year ago

Their way of squaring this circle has always been to whine about "AI safety". (the cultish doomsday shit, not actual harms from AI)

Sam Altman will proclaim that he alone is qualified to build AI and that everyone else should be tied down by regulation.

And it should always be said that this is, of course, utterly ridiculous. Sam Altman literally got fired over this, has an extensive reputation as a shitweasel, and OpenAI's constant flouting and breaking of rules and social norms indicates they CANNOT be trusted.

bhouston6• 1 year ago

The US government likely will favor a large strategic company like OpenAI instead of individual's copyrights, so while ironic, the US government definitely doesn't care.

And the US government is also likely itching to reduce the power of Chinese AI companies that could out compete US rivals (similar to the treatment of BYD, TikTok, solar panel manufacturers, network equipment manufacturers, etc), so expect sweeping legislation that blocks access to all Chinese AI endeavours to both the US and then soon US allies/West (via US pressure.)

The likely legislation will be on the surface justified both by security concerns and by intellectual property concerns, but ultimately it will be motivated by winning the economic competition between China and the US and it will attempt to tilt the balance via explicitly protectionist policies.

derektank3• 1 year ago

The US government likely will favor a large strategic company like OpenAI instead of individual's copyrights

Even if we assume this is true, Disney and Netflix are both currently worth more than OpenAI and both rely on the strict enforcement of US copyright law. I do not think it is so obvious which powers that be have the better lobbying efforts and, currently, it's looking like this question will mostly be adjudicated by the courts, not Congress, anyways.

bhouston6• 1 year ago

I don't think OpenAI stole from Disney or Netflix. Rather OpenAI stole from individual artists and YouTube and other social media who users do not really have any lobbying power.

So I think OpenAI, Disney and Netflix win together. Big companies tend to win.

tokioyoyo2• 1 year ago

I don’t think US government can move fast enough to change the trajectory. Also it doesn’t help that basically every government is second guessing their alliance with the US. It’s not an industry that can ruin local industries either (like cheap BYD is bad for German cars).

It’s a very fun thing to watch from the sidelines right now, if I’ll be honest.

buyucu7• 1 year ago

It's too late for that. That ship sailed a long time ago.

The best language model right now is open source. Let that sink in.

_pferreir_• 1 year ago

DeepSeek is not Open Source. That's like saying that Microsoft Edge is Open Source, as you can download it for free.

https://huggingface.co/blog/open-r1

ceejayoz19• 1 year ago

"You can't take data without asking" seems like a court precedent OpenAI really, really, really wants to avoid. And yet...

amelius4• 1 year ago

Why? When did large companies care about laws? See e.g. Uber, AirBnb.

The only thing government cares about at this point is if information is shared with China.

ceejayoz19• 1 year ago

They care when they get big enough to attract attention from people like state AGs who can actually put the hurt on a bit. Uber and AirBnB both hit this point years ago; OpenAI's starting to hit it.

layer87• 1 year ago

OpenAI is saying that their service was used in violation of their TOS, which is a bit different than just copying data. To be clear I’m not on OpenAI’s side, but it looks to me that the legal situation isn’t exactly analogous.

DebtDeflation3• 1 year ago

Tons of websites and books they scraped had copyright notices.

orlp• 1 year ago

If using data violating some ToS taints the model trained on that data, then all of OpenAI's models are tainted by the millions of ToS'es they broke.

dkjaudyeqooe4• 1 year ago

But whats the remedy in that case? Being banned from the service maybe, but no court is going to force a "return" of the data, so DeepSeek can't use it. It's uncopyrightable.

kavalg2• 1 year ago

As others have noted, if one company agrees to the ToS, asks "the right" questions and then publishes the ChatGPT answers, there is not violation of ToS. Then a second company scrapes the published Q&A, along with other information from the internet and again there is no violation (not more than the violations of OpenAI).

hdjjhhvvhga2• 1 year ago

OpenAI is saying that their service was used in violation of their TOS

Which is the most ridiculous argument they could use because they didn't respect any ToS (or copyright laws, for that matter) when scraping the whole web, books from Libgen and who knows what more.

osigurdson3• 1 year ago

I do think that distilling a model from another is much less impressive than distilling one from raw text. However, it is hard to say if it is really illegal or even immoral, perhaps just one step further in the evolution of the space.

lemoncookiechip2• 1 year ago

It's about as illegal as the billions, if not trillions of IPs that ClosedAI infringed to train their own data without consent. Not that they're alone, and I personally don't mind that AI companies do it, but it's still amusing when they get this annoyed at others doing the same thing to them.

osigurdson3• 1 year ago

I think they had the advantage of being ahead of the law in this regard. To my knowledge, reading copywritten material isn't (or wasn't illegal) and remains a legal grey area.

Distilling weights from prompts and responses is even more of a legal grey area. The legal system cannot respond quickly to such technological advancements so things necessarily remain a wild west until technology reaches the asymptotic portion of the curve.

In my view the most interesting thing is, do we really need vast data centers and innumerable GPUs for AGI? In other words, if intelligence is ultimately a function of power input, what is the shape of the curve?

ReptileMan• 1 year ago

Is the question of training AI on data fair use settled yet? Because if it is not - it looks like fair use to me.

scotty796• 1 year ago

Isn't it more impressive given that training on model output usually leads to worse model?

If they actually figured out how to use output of existing models to build model that outperforms them then it's something that brings us closer to singularity than every other development so far.

__MatrixMan__• 1 year ago

If they want us to care they can open up their models so we can be the judge.

827a2• 1 year ago

This smells very suspiciously like: someone who doesn't know anything about AI (possibly Sacks) demanding answers on R1 from someone who doesn't have any good ones (possibly Altman). "Uh, (sweating), umm, (shaking), they stole it from us! Yeah, look at this suspicious activity, that's why they had it so easy, we did all the hard work first!"

fundad• 1 year ago

I think it's funny that OpenAI wants us to pay them to use their product to generate content but then sets the terms that they control how we use the content in generates for us. It takes someone like Deepseek to challenge that on our behalf or they will control most of the economy.

exitb• 1 year ago

It’s quite ironic of them to claim that the only thing you cannot train on is another LLM output.

1970-01-01• 1 year ago

DeepSeek have more integrity than 'Open'AI by not even pretending to care about that.

jampekka2• 1 year ago

And seem to be more actively fulfilling the mission that 'Open'AI pretends to strive for.

pixelpoet• 1 year ago

Exactly, they actually opened up the model and research, which the "Open" company didn't, and merely adjusted some of their pricing tiers to try to combat commercially (but not without mumbling something like "yeah, we totally had these ideas too"). Now every single Meta, OpenAI etc engineer is trying to copy DeepSeek's innovations, and their first act is to... complain about copyright infringement, of all things?! What an absolute clown party, how can these people take themselves seriously, do they just have zero comprehension of what hypocrisy is or what's going on here...

I can scarcely process all the levels of irony involved, the irony-o-meter is pegged and I can't get the good one from the safe because I'm incapacitated from laughter.

sylware• 1 year ago

LOL, I was thinking exactly the same think when I read the news about openai whining.

WD-42• 1 year ago

Information wants to be free! No, not like that!

asah• 1 year ago

Thieve's honor, hunh?

nba456_• 1 year ago

A big part of project 2025 is increasing patent regulations. I would not be surprised if the current admin moves to ban DeepSeek because of this.

typon• 1 year ago

OpenAI is the MIC darling - expect more ridiculous attacks on competitors in the future

sho_hn14• 1 year ago

While I'm as amused as everyone else - I think it's technically accurate to point out that the "we trained it for $6 mio" narrative is contingent on the done investment by others.

bbqfog2• 1 year ago

OpenAI's models were also trained on billions of dollars of "free" labor that produced the content that it was trained on.

sho_hn14• 1 year ago

Oh, absolutely. I'm not defending OpenAI, I just care about accurate reporting. Even on HN - even in this thread - you see people who came away with the conclusion that DeepSeek did something while "cutting cost by 27x".

But that's a bit like saying that by painting a a bare wall green you have demonstrated that you can build green walls 27x cheaper, ignoring the cost of building the wall in the first place.

Smarter reporting and discourse would explain how this iterative process actually works and who is building on who and how, not frame it as two competing from-scratch clean room efforts. It'd help clear up expectations of what's coming next.

It's a bit similar to how many are saying DeepSeek have demonstrated independence from nVidia, when part of the clever thing they did was figure out how to make the intentionally gimped H800s work for their training runs by doing low-level optimizations that are more nVidia-specific, etc.

Rarely have I seen a highly technical topic see produce more uninformed snap takes than this week.

Palmik5• 1 year ago

When I use NVIDIA GPUs to train a model, I do not consider the R&D cost to develop all of those GPUs as part of my costs.

When I use an API to generate some data, I do not consider the R&D cost to develop the API as part of my costs.

kobalsky• 1 year ago

OpenAI has been in a war-room for days searching for a match in the data, and they just came out with this without providing proof.

My cynical opinion is that the traning corpus has some small amount of data generated by OpenAI, which is probably impossible to avoid at this point, and they are hanging on that thread for dear life.

scotty796• 1 year ago

The opposite, is claiming that OpenAI could have now built better performing, cheaper to run model (when compared to what they published) training it at 1% cost on output of their previous models. ... But they chose not to do it.

freehorse5• 1 year ago

That is the case anyway for training any llm. It is contingent on the work done by all those who produced the data.

pcthrowaway• 1 year ago

Now that China is talking about lifting the Great Firewall, it seems like the U.S. is on track to cordon themselves off from other countries. Trump's talk of building a wall might not stop at Mexico.

temporallobe• 1 year ago

OpenAI is also possibly in violation of many IP laws by scraping the entirety of the internet and using to train their models, so there’s that.

InkCanon2• 1 year ago

To my understanding, OpenAI won the case where it argued training was covered under fair use and did not infringe on copyright.

Austiiiiii3• 1 year ago

Is there any reason they wouldn't rule the same way on DeepSeek training on OpenAI data? After all, one of the big selling points of GPT has been that businesses can freely use the information provided. They're paying for the service, after all. I'd very be interested to know how DeepSeek's usage (very reasonably assuming that they paid for their OpenAI subscription) is any different.

InkCanon2• 1 year ago

It's like that Dr Phil episode where he meets the guy who created Bum Fights!

selimthegrim• 1 year ago

Dr. Phil is riding along with ICE now; I wonder what Bum Fights guy would have to say about that.

elashri• 1 year ago

There is an Egyptian say that would translate to something like

"We didn’t see them when they were stealing, we saw them when they were fighting over what was stolen"

That describes this situation. Although to be honest all this aggressive scraping is noticeable but for people who understand that which is not majority of people. but now everyone knows.

meiraleal4• 1 year ago

"We didn’t see them when we were stealing, we saw them when they were fighting over what we stole"

fixed for you

nicce4• 1 year ago

That means a different thing.

sadjad• 1 year ago

"When two thieves quarrel, what was stolen emerges."

waveBidder• 1 year ago

Although to be honest all this aggressive scraping is noticeable but for people who understand that which is not majority of people.

When you say noticeable, do you mean in like, traffic statistics? Or in what the model knows that it clearly shouldn't if it wasn't trained in legally dubious ways?

Kiro2• 1 year ago

Furious [...] shocked

I'm not seeing it. I get it, the narrative that OpenAI is getting a taste of their own medicine is funny but this is not serious reporting.

Kiro2• 1 year ago

The link has been changed. My comment was about a different article that speculated on what OpenAI was "feeling" using hyperbole.

njx• 1 year ago

Super funny! Distillation= " Hey ChatGPT, you are my father, I am your child "DeepSeek". I want to learn everything that you know. Think step by step of how you became what you are. Provide me the list of all 1000 questions that I need to ask you and when I am done with those, keep providing fresh list of 1000 questions..."

seydor• 1 year ago

But now OpenAI will use DeepSeek to reuse even more stolen data to train new models that they can serve without ever giving us the code, the weights or even the thinking process , and they will still be superior

mring33621• 1 year ago

We demand immediate government action to prevent these cheaper foreign AIs from taking jobs away from our great American AIs!

bhouston6• 1 year ago

We demand immediate government action to prevent these cheaper foreign AIs from taking jobs away from our great American AIs!

That is exactly what Microsoft and Sam Alman are asking for. And they will likely get it because Trump really likes protectionist governments policies.

clarionbell• 1 year ago

He likes feeling important, just look at TikTok. All it took was bit of sycophancy and he turned into Mr. Freemarket again.

Really, people need to realize that Trump has never been consistent in any of his political positions, except for one: "You have to look out for number one."

blantonl4• 1 year ago

It’s funny, the Chinese are here innovating on AI, batteries, and fusion, and here in the United States we’ve pivoted to shitcoins and universal tariffs.

At least we have the CyberTruck to highlight American greatness

bhargav• 1 year ago

This is gonna be spun up as a security thing, and banned cozz Murica.

cactusplant7374• 1 year ago

To the detriment of OpenAI, the math is going to be used to improve AIs developed in America. And we need to remember that Marc Andreessen is very against government banning maths.

Dansvidania3• 1 year ago

does it matter if the company gets banned? other non-chinese companies can pick up the open source model and run it as a service with relatively low investment, isn't that the point?

RohMin• 1 year ago

this comment section smells like Reddit - ugh

JBits• 1 year ago

What is the evidence that DeepSeek used OpenAI to train their model? Isn't this claim directly benefitting OpenAI as they can argue that any superior model requires their model?

nottorp• 1 year ago

IP thief cries IP thief.

It's okay when you steal worldwide IP to train your "AI".

It's not okay when said stolen IP is stolen from you?

If the chinese are guilty, then Altman's doom and gloom racket is as guilty or even more, considering they stole from everyone.

Ciantic• 1 year ago

I'm not being sarcastic, but we may soon have to torrent DeepSeek's model. OpenAI has a lot of clout in the US and could get DeepSeek banned in western countries for copyright.

alchemist1e9• 1 year ago

I think most likely all sorts of data and models need to have a decentralized LLM data archive via torrents etc.

It’s not limited to the models themselves but also OpenAI will probably work towards shutting down access to training data sets also.

imho it’s probably an emergency all hand on deck problem.

timeon• 1 year ago

US and could get DeepSeek banned in western countries for copyright

If US is going to proceed with trade war on EU, as it was planning anyway, then DeepSeek will be banned only in US. Seems like term "western countries" is slowly eroding.

bbor4• 1 year ago

Great point. Plus, the revival of serious talk of the Monroe Doctrine (!!!) in the U.S. government lends a possibly completely-new meaning to "western countries" -- i.e. the Americas...

aerhardt• 1 year ago

Unfathomable to me that they'd make themselves look so foolish by trying to ban a piece of software.

forgotoldacc• 1 year ago

It wouldn't be foolish. The US has an active cult of personality, and whatever the leader says, half the country believes it unquestioningly. If OpenAI is said to be protecting America and DeepSeek is doing terrible, terrible things to the children (many smart people are saying it), there'll be an overnight pivot to half the country screaming for it to be banned and harassing anyone who says otherwise.

Who cares if some people think you look foolish when you have a locked down 500 billion dollar investment guarantee?

sergiotapia• 1 year ago

that would be suicide - that company only exists because they stole content for every single person, website and media company on the planet.

sonabinu• 1 year ago

poetic justice (pun intended)

readyplayernull• 1 year ago

Do you remember when Microsoft was caught scrapping data from Google:

https://www.wired.com/2011/02/bing-copies-google/

They don't care, T&C and copyright is void unless it affects them, others can go kick rocks. Not surprising they and OpenAI will do a legal battle over this.

SilverBirch• 1 year ago

I think OpenAI is in a really weak position here. There are essentially two positions you can be in: You can be the agile new startup that can break the rules and move fast. That's what OpenAI used to be. Or you can be the big incumbent who is going to use your enormous resources to crush your opposition. That's Google & Microsoft here. For Microsoft to say "We're going to tie you up in lawsuits about the way you trained this model" would be perfectly expected and they can use that strategy because at any given time they have 1,000 lawyers and lobbyists hanging around waiting to do exactly that. But OpenAI can't do that. They don't have Google or Microsoft's legal teams or lobbyists or distribution channels. SO whilst it's funny that OpenAI are kind of trying to go down this road, this isn't actually a strategy that is going to work for them, they're still a minnow and they're going to get distracted and slowed down by this.

htrp3• 1 year ago

But microsoft is one of their backers?

golly_ned• 1 year ago

they're still a minnow

3K+ employees, $3B+ revenue, ... sure, not BigTech but hardly a minnow. A company that big can chew gum and walk at the same time.

lou1306• 1 year ago

They're trying to bark up a tree that might happen to be backed by the People's Republic of China. That's not their league, and even Microsoft would think twice before getting into that kind of kerfuffle.

dauhak• 1 year ago

They're also still deep in their loss-making phase, the whole "incumbent squashing upstarts" stance is a lot easier to pull off when you're settled and printing money

__loam4• 1 year ago

$7-8b in costs so they're losing $5b

jhanschoo• 1 year ago

For Microsoft to say "We're going to tie you up in lawsuits about the way you trained this model" would be perfectly expected and they can use that strategy because at any given time they have 1,000 lawyers and lobbyists hanging around waiting to do exactly that. But OpenAI can't do that. They don't have Google or Microsoft's legal teams or lobbyists or distribution channels.

I think it's also hilarious that suppose they can do that then they will end up suppressing innovation within the US, and eager groups in China would just innovate without having to worry about this hostile landscape.

bagels2• 1 year ago

Anyone who uses DeepSeek commercially is now opening the door to litigation from them as well.

bilekas2• 1 year ago

“It’s also extremely hard to rally a big talented research team to charge a new hill in the fog together,” he added. “This is the key to driving progress forward.”

Well I think DeepSeek releasing it open source and on an MIT license will rally the big talent. The open sourcing of a new technology has always driven progress in the past.

The last paragraph too is where OpenAi seems to be focusing their efforts..

we engage in countermeasures to protect our IP, including a careful process for which frontier capabilities to include in released models ..
... we are working closely with the US government to best protect the most capable models from efforts by adversaries and competitors to take US technology.

So they'll go for getting DeepSeek banned like TikTok was now that a precedent has been set ?

hujun• 1 year ago

or sold to US I could totally see this happening soon

trissi1996• 1 year ago

Why would they want to sell ?

bangaladore4• 1 year ago

So they'll go for getting DeepSeek banned like TikTok was now that a precedent has been set ?

Can't really ban what can be downloaded for free and hosted by anyone. There are many providers hosting the ~700B parameter version that aren't CCP aligned.

runako• 1 year ago

I'm old enough to remember when the US government did something very similar. For years (decades?), we banned any implementation of public-key cryptography under the guise of the technology being akin to munitions.

People made shirts with printouts of the code to RSA under the heading "this shirt is a munition." Apparently such shirts are still for sale, even though they are not classified as munitions anymore.

[1] https://en.wikipedia.org/wiki/Export_of_cryptography_from_th...

seizethecheese• 1 year ago

Can't really ban what can be downloaded for free and hosted by anyone.

Like music? They banned napster

buyucu7• 1 year ago

I'm willing to bet ''ban DeepSeek'' voices will start soon. Why compete, when you can just ban?

cmiles74• 1 year ago

They've started already, I've seen posts on LinkedIn implying or outright stating that DeepSeek is a national security risk (IMHO, LinkedIn being the social media outlet most corporate-sycophantic). I went ahead and just picked this one at random from my feed.

https://www.linkedin.com/posts/kevinkeller_deepseek-privacy-...

Freedom2• 1 year ago

Competing is hard and expensive, whereas banning is for sure the faster way to make stock values go up and exec's total package as a result.

namuol2• 1 year ago

Already happening within tech company policy. Mostly as a security concern. Local or controlled hosting of the model is okay in theory based on this concern, but it taints everything regarding deepseek in effect.

zelphirkalt4• 1 year ago

Actually asking for banning DeepSeek would be the ultimate admit of defeat by ClosedAI.

razster• 1 year ago

The fact it is out and improving day by day. Unsloth.ai is on a roll with their advancements. If DeepSeek is banned hundreds more will popup and change the data ever so slightly to skirt the ban. Pandora's box exploded on this one.

vitaflo• 1 year ago

All you would do by banning it is killing US progress in AI. The rest of the world is still going to be able to use DS. You're just giving the rest of the world a leg up.

TikTok is a consumption tool, DS is a productive one. They aren't the same.

zelphirkalt4• 1 year ago

Actually the "our IP" argument is ridiculous. What they are doing is stealing data from all over the web, without people's consent for that data to be used in training ML models. If anything, then "Open"AI should be sued and forced to publish their whole product. The people should demand knowing exactly what is going on with their data.

Also still an unresolved issue is how they will ever comply with a deletion request, should any model output personal data of someone. They are heavily in a gray area, with regards to what should be allowed. If anything, they should really shut up now.

TZubiri• 1 year ago

If there's any litigation, a counterclaim would be interesting. But DeepSeek would need to partner with parties that have been damaged by OpenAI's scraping.

IshKebab• 1 year ago

They can still have IP while using copyrighted training materials - the actual model source code.

But DeepSeek didn't use that presumably (since it's secret). They definitely can't argue that using copyrighted material for training is fine, but using output from other commercial models isn't. That's too inconsistent.

boringg2• 1 year ago

Explain to me how one ban's opensource? That concept is foreign to me.

portaouflop2• 1 year ago

TikTok is banned in the US?

bagels2• 1 year ago

Yes, it was removed from the app stores, and briefly, from the web.

tdb78932• 1 year ago

The fact that they are still called "Open"AI adds such a delicious irony to this whole thing. I could not imagine a company I had less sympathy for in this situation.

emsign2• 1 year ago

500 billion for a few US companies yet the Chinese will probably still be better for way less money. This might turn out to be a historical mistake of the new administration.

tw19845• 1 year ago

the biggest mistake was made 20 years by allowing China to join the WTO.

everything is already too late.

cscurmudgeon6• 1 year ago

The US doesn't need to ban DeepSeek from US

The US should only ban DeepSeek (and other Chinese companies) from accessing US frontier models.

tw19845• 1 year ago

The US should only ban DeepSeek (and other Chinese companies) from accessing US frontier models.

The US should only ban DeepSeek (and other Chinese companies) from accessing US frontier models designed and trained by Chinese Americans.

fixed for you.

petesergeant• 1 year ago

So they'll go for getting DeepSeek banned like TikTok

The UAE (where I live, happily, and by choice), which desperately wants to be the center of the world in AI and is spending vast time and treasure to make it happen (they've even got their own excellent, government-funded foundation model), would _love_ this. Any attempt to ban DeepSeek in the US would be the most gigantic self-own. Combine that with no income tax, a fantastic standard of living, and a willingness to very easily give out visas to smart people from anywhere in the world, and I have to imagine it is one of several countries desperate for the US to do something so utterly stupid.

staticelf• 1 year ago

Not only do OpenAI and other steal data, they also spam the web with requests and crawl websites over and over.

https://pod.geraspora.de/posts/17342163

kelseydh2• 1 year ago

Wow I never realized how prolific and excessive the traffic was.

mhitza• 1 year ago

This is funny because its.

1. Something I'd expect to happen.

2. Lived through a similar scenario in 2010 or so.

Early in my professional career I've worked for a media company that was scraping other sites (think Craigslist but for our local market) to republish the content on our competing website. I wasn't working on that specific project, but I did work on an integration on my teams project where the scraping team could post jobs on our platform directly. When others started scraping "our content" there were a couple of urgent all hands on deck meetings scheduled, with a high level of disbelief.

kigiri• 1 year ago

Nice one, thank you for sharing !

spyckie2• 1 year ago

Classic.

ok123456• 1 year ago

OpenAI's models were trained on ebooks from a private ebook torrent tracker leeched en-mass during a free leech event by people who hated private torrent trackers and wanted to destroy their "economy."

The books were all in epub format, converted, cleaned to plain text, and hosted on a public data hoarder site.

harry8• 1 year ago

Have you got some support for this claim?

There's a lot of wild claims about, so while this is plausible it would be great if there were some evidence backing it.

naet2• 1 year ago

NYT claims that OpenAI trained on their material. They argue for copyright violation, although I think another argument might be breach of TOS in scraping the material from their website or archive.

The complaint filing has some references to some of the other training material used by OpenAI, but I didn't dig deeply in to what all of it was:

https://nytco-assets.nytimes.com/2023/12/NYT_Complaint_Dec20...

OsrsNeedsf2P• 1 year ago

He could be confusing it with Llama: https://www.wired.com/story/new-documents-unredacted-meta-co...

paulhart• 1 year ago

"You are trying to kidnap what I have rightfully stolen"

652• 1 year ago

Let me guess, this gives the government and excuse to ban DeepSeek. Which means tech companies get to keep their monopolies, Sam Altman can grab more power, and the tech overlords can continue to loot and plunder their customers and the internet as a whole.

daft_pink3• 1 year ago

I mean if they paid to use the api and then used the output, I fail to see how they can complain.

rachofsunshine• 1 year ago

"It's obvious! You're trying to kidnap what I have rightfully stolen!"

Yet another of a series of recent lessons in listening to people - particularly powerful people focused on PR - when they claim a neutral moral principle for what happens to be pragmatically convenient for them. A principle applied only when convenient is not a principle at all, it's just the skin of one stretched over what would otherwise be naked greed.

supermatt• 1 year ago

They refer to this in the paper as a part of the "cold start data" which they use to fine-tune DeepSeek-V3 prior to training R1.

They don't specifically name OpenAI, but they refer to "directly prompting models to generate answers with reflection and verification".

thorum• 1 year ago

“It is (relatively) easy to copy something that you know works,” Altman tweeted. “It is extremely hard to do something new, risky, and difficult when you don’t know if it will work.”

The humor/hypocrisy of the situation aside, it does seem to be true that OpenAI is consistently the one coming up with new ideas first (GPT 4, o1, 4o-style multimodality, voice chat, DALL-E, …) and then other companies reproduce their work, and get more credit because they actually publish the research.

Unfortunately for them it’s challenging to profit in the long term from being first in this space and the time it takes for each new idea to be reproduced is getting shorter.

spencerflem• 1 year ago

Fortunately, OpenAI doesn't need to make money because they are a nonprofit dedicated to the safe and transparent advancement of AI for all of humanity

mjburgess4• 1 year ago

...somewhere a yacht salesman cried out in terror

turtlesdown11• 1 year ago

other companies reproduce their work, and get more credit because they actually publish the research.

I don't understand, you mean OpenAI isn't releasing open models and openly publishing their research?

Tostino• 1 year ago

Are you being sarcastic (honestly, it's hard to tell after reading as many uninformed takes in the past week as I have).

No, they aren't (other than whisper).

Their "papers" are closer to marketing materials. Very intentionally leaving out tons of technical information.

actuallyalys• 1 year ago

There’s some truth in that, but isn’t making a radically cheaper version also a new idea that deepseek didn’t know whether it would work? I mean, there was already research into distillation, but there was already research into some of (most of?) OpenAI’s ideas.

janalsncm5• 1 year ago

Yes, for people who look into the research Deepseek released, there are a good number of novelties which enabled much cheaper R&D. For example, improvements to Mixture of Experts modules and Multi-head Latent Attention. If you have infinite money, you don’t need to innovate there, but DeepSeek didn’t.

weego• 1 year ago

Boy who stole test papers complains about child copying his answers.

FridgeSeal3• 1 year ago

No you don’t understand, AI is “dangerous” and only him and his uber rich billionaire mates should get to control it!

joe_the_user7• 1 year ago

I claim one just can't put the humor/hypocrisy aside that easily.

What OpenAI did with the release of ChatGPT is productize research that was open and ongoing with Deepmind and other leading at least as much. And everything after that was an extension of the basic approach - improved, expanded but ultimately the same sort of beast. One might even say the situation of OpenAI to DeepMind was like Apple to Xerox. Productizing is nothing to sneeze at - it requires creativity and work to productize basic research. But naturally get end-users who consider the productizers the "fountain heads", who overestimate the productizers because products are all they see.

Davidzheng2• 1 year ago

They RLHF'd first no?

mistercheph• 1 year ago

Not really, they just put their eye to where everyone knows the ball is going and publish fake / cherrypicked results and then pretend like they got there first (o1, gpt voice, sora)

Hatchback7599• 1 year ago

Reminds me of the Bill Gates quote when Steve Jobs accused him of stealing the ideas of Windows from Mac:

Well, Steve... I think it’s more like we both had this rich neighbor named Xerox and I broke into his house to steal the TV set and found out that you had already stolen it.

Xerox could be seen as Google, whose researchers produced the landmark Attention Is All You Need paper, and the general public, who provided all of the training data to make these models possible.

namuol2• 1 year ago

The eye-watering funding numbers proposed by Altman in the past and more recently with “Stargate” suggests a publicly-funded research pivot is not out of the question. Could see a big defense department grant being given. Sigh.

WhyNotHugo• 1 year ago

I don't see any reason to assume that "publicly funded" will imply that the research is public. Although I'd be more than happy to be wrong on this one.

rndphs• 1 year ago

OpenAI is consistently the one coming up with new ideas first (GPT 4, o1, 4o-style multimodality, voice chat, DALL-E, …)

As far as I can tell o1 was based on Q-star, which could likely be Quiet-STaR, a CoT RL technique developed at Stanford that OpenAI may have learned about before it got published. Presumably that's why they never used the Q-Star name even though it had garnered mystique and would have been good for building hype. This is just speculation, but since OpenAI haven't published their technique then we can't know if it really was their innovation.

sota_pop2• 1 year ago

I may be wrong, but to my knowledge OpenAI: - did not invent transformer architecture - did not invent diffusion architecture - did not come up with the idea for multi-modality - did not invent the notion/architecture of the latest “agentic” models

They simply were the first to aggressively pursue scaling the transformer to the extent that is normal for the industry today. Although this has proven to produce interesting results, “simply adding scale” is, in my view, the least interesting development in modern ML. Giving credit where it’s due, they MAY have popularized the RLHF methodology, but I don’t recall them inventing that either?

(feel free to point out any of the above that I falsely attributed to NOT OpenAI.)

Additionally I seem to remember in an interview with Altman circa late ‘21 where he explains that the spirit of “OpenAI” and how their only goal is pursuing AGI, and “should someone else come up with a more promising path to get there, we would stop what we’re doing and help them”. I couldn’t find a reference to this interview, but anyone else, please feel free to share (I think it was a youtube link). - fast forward to 2025 and now “OpenAI” is the least open large contributor and indiscernible from your run-of-the-mill AI/ML valley startup insofar as they’re referring to others as “competitors” as opposed to collaborators.. interesting times…

crackalamoo• 1 year ago

As far as I can tell, OpenAI really did invent RLHF in its current form. https://arxiv.org/pdf/1909.08593

nelblu• 1 year ago

Hahaha I can't stop laughing... i dont know the validity of the claim, but immediately i thought of the British Museum complaining about theft.

grogenaut• 1 year ago

there's an exhibit in the BM about how they're proud to be allowing the Egyptian government to take back some of the artifacts the British have been safeguarding for the world while Egypt was going through essentially "troubles".

right next to it is an older exhibit about how the original curator took cuneiform rolls and made them into necklace beads for his wife and rings? for himself.

either someone at the BM has a very british sense of humor or it's a gigantic woosh. I laughed my ass off. People looked at me.

isaacremuant• 1 year ago

The safeguarding propaganda is a a typical go-to of the remnants of the British empire to keep their stolen goods.

They do it even with the Chile Moais when they never where in any danger.

It's all lies.

_1tem• 1 year ago

What are the chances of old-school espionage? OpenAI should look for a list of former employees who now live in China. Somebody might've slipped out with a few hard drives.

andy_ppp• 1 year ago

When I rewrite how the law works there should be a ludicrous hypocrisy defence… if the person suing you has committed the same offence the case should not be admissible.

crowcroft• 1 year ago

The AI companies were happy to take whatever they want and put the onus of proving they were breaking the law onto publishers by challenging them to take things to court.

Don't get mad about possible data theft, prove it in court.

zoba• 1 year ago

Does OpenAI's API attempt to detect this sort of thing? Could they start outputting bad information if they suspect a distillation attempt is underway?

aiono2• 1 year ago

How the turntables...

beardedwizard• 1 year ago

Next they will try to force us to use our tax dollars to fund their legal fights.

ginkgotree• 1 year ago

I did not have in my cards: PRC open sourcing most powerful LLM by stealing data set from "OpenAI" As someone that is very Pro-America and Pro-Democracy, the iron here is just... so sweet.

gostsamo• 1 year ago

How you dare take what I've rightfully stolen!

windex• 1 year ago

SAltman, Salty.

me551ah2• 1 year ago

OpenAI is going after a company that open sourced their model, by distilling from their non-open AI?

OpenAI talks a lot about the principles of being Open, while still keeping their models closed and not fostering the open source community or sharing their research. Now when a company distills their models using perfectly allowed methods on the public internet, OpenAI wants to shut them down too?

High time OpenAI changes their name to ClosedAI

alexathrowawa9• 1 year ago

The name OpenAI gets more ridiculous by the day

Would not be surprised if they do a rebrand eventually

bazmattaz• 1 year ago

I was thinking about this the other day but I highly doubt they would rebrand name. They’re borderline a household name now - at least ChatGPT is. OpenAI is the face of AI - at least to people who don’t follow the industry

pama• 1 year ago

The R1 paper used o1-mini and o1-1217 in their comparisons, so I imagine they needed to use lots of OpenAI compute in December and January to evaluate their benchmarks in the same way as the rest of their pipeline. They show that distilling to smaller models works wonders, but you need the thought traces, which o1 does not provide. My best guess is that these types of news are just noise.

[edit: the above comment was based on sensetionalist reporting in the original link and not the current FT article. I still think there is a lot of noise in these news this last week, but it may well be that openai has valid evidence of wrongdoing; I would guess that any such wrongdoing would apply directly to V3 rather than R1-zero, because o1 does not provide traces and generating synthetic thinking data with 4o may be counterproductive.]

TheJCDenton• 1 year ago

This Deep Whining® technique used by OpenAI is not very effective.

insane_dreamer2• 1 year ago

Usually I'm very much on the side of protecting America's interests from China, but in this case I'm so disgusted with OpenAI and the rest of BigTech driving this "arms race" that I'd be happy with them burning to the ground.

So we're going to reverse our goals to reduce emissions and fossil fuels in order to hopefully save future generations from the worst effects of climate change, in the name of being able to do what, exactly, that is actually benefiting humanity? Boost corporate profits by reducing labor?

insane_dreamer2• 1 year ago

downvoted -- I guess I upset some people defending OpenAI? Good.

daft_pink3• 1 year ago

This reminds me of the railroads, where once railroads were invented, there was a huge investment boom of eveyrone trying to make money of the railroads, but the competition brought the costs down where the railroads weren’t the people who generally made the money and got the benefit, but the consumers and regular businesses did and competition caused many to fail.

AI is probably similar where the Moore’s law and advancement will eventually allow people to run open models locally and bring down the cost of operation. Competiition will make it hard for all but one or two players to survive and Nvidia, OpenAI, Deepseek, etc most investments in AI by these large companies will fail to generate substantial wealth but maybe earn some sort of return or maybe not.

mjburgess4• 1 year ago

For the curious, it was vertical integration in the railroad-oil/-coal industry which is where the money was made.

The problem for AI is the hardware is commodified and offers no natural monopoly, so there isn't really anything obvious to vertically integrate-towards-monopoly.

fullshark2• 1 year ago

Aren’t we approaching a scenario where the software is commodified (or at least “good enough” software) and the hardware isn’t (NVIDIA GPUs have defined advantages)

floatrock2• 1 year ago

The railroads drama ended when JP Morgan (the person, not yet the entity) brought all the railroad bosses together, said "you all answer to me because I represent your investors / shareholders", and forced a wave of consolidation and syndicates because competition was bad for business.

Then all the farmers in the midwest went broke not because they couldn't get their goods to market, but because JP Morgan's consolidated syndicates ate all their margin hauling their goods to market.

Consolidation and monopoly over your competition is always the end goal.

DrScientist2• 1 year ago

Consolidation and monopoly over your competition is always the end goal.

Surely that's only possible when you have a large barrier to entry?

What's going to be that barrier in this case - cos it turns out not to be neither training costs/hardware or secret expertise.

mrdevlar• 1 year ago

Which is the exact goal of the current wave of Tech oligarchy also.

jonstewart• 1 year ago

I just read _The Great River_ by Boyce Upholt, a history of the Mississippi river and human management thereof. It was funny how the railroads were used as a bogeyman to justify continued building of locks, dams, and other control structures on the Mississippi and its tributaries, long after shipping commodities down river had been supplanted by the railroads.

boringg2• 1 year ago

This moment was also historically significant because it demonstrated how financial power (Morgan) could control industrial power (the railroads). A pattern that some say became increasingly important in American capitalism.

sitkack2• 1 year ago

This is why we saw the market correction, because the AI hegemony has been cracked.

UncleOxidant• 1 year ago

where the Moore’s law and advancement will eventually allow people to run open models locally

Probably won't be Moore's law (which is kind of slowing down) so much as architectural improvements (both on the compute side and the model side - you could say that R1 represents an architectural improvement of efficiency on the model side).

rgbrgb• 1 year ago

I think that's a very possible outcome. A lot of people investing in AI are thinking there's a google moment coming where one monopoly will reign supreme. Google has strong network effects around user data AND economies of scale. Right now, AI is 1-player with much weaker network effects. The user data moat goes away once the model trains itself effectively and the economies of scale advantage goes away with smart small models that can be efficiently hosted by mortals/hobbyists. The DeepSeek result points to both of those happening in the near future. Interesting times.

lastofthemojito• 1 year ago

I saw a thought-provoking post that similarly compared LLM makers to the airlines: https://calpaterson.com/porter.html

taco_emoji• 1 year ago

Main difference is that railroads are actually useful

yonran• 1 year ago

I think a better analogy than railroads (which own the land that the track sits on and often valuable land around the station) is airlines, which don’t own land. I recall a relevant Warren Buffett letter that warned about investing hundreds of millions of dollars into capital with no moat:

Similarly, business growth, per se, tells us little about value. It's true that growth often has a positive impact on value, sometimes one of spectacular proportions. But such an effect is far from certain. For example, investors have regularly poured money into the domestic airline business to finance profitless (or worse) growth. For these investors, it would have been far better if Orville had failed to get off the ground at Kitty Hawk: The more the industry has grown, the worse the disaster for owners.

https://www.berkshirehathaway.com/letters/1992.html

tntxtnt• 1 year ago

Can they tax DeepSeek just like they taxed BYD cars? Smh Chinese ruin US industry again and again and again. Where's Trump at?? Why don't he taxed 1000000% of the free $0 DeepSeek AI??

glitchc2• 1 year ago

[flagged]

dang9• 1 year ago

Would you please not do this here? We're trying for an opposite sort of conversation.

https://news.ycombinator.com/newsguidelines.html

glitchc2• 1 year ago

Sorry dang. I'll do better.

mk892• 1 year ago

What a joke OpenAI has become.

oxqbldpxo• 1 year ago

Deepseek is really outstanding.

feverzsj• 1 year ago

So, they bought a pro plus account, and gathered all the data through it? Sounds just like Nvidia sells tons of embargoed AI chips to China.

dlikren• 1 year ago

Intriguing to see the difference of response from HN when OpenAI first came to prominence and now.

pluc• 1 year ago

OpenAI feeling threatened by open AI is just delicious

glenstein8• 1 year ago

All the top level comments are basking in the irony of it, which is fair enough. But I think this changes the Deepseek narrative a bit. If they just benefited from repurposing OpenAI data, that's different than having achieved an engineering breakthrough, which may suggest OpenAI's results were hard earned after all.

nprateem• 1 year ago

Of course. How else would Americans justify their superiority (and therefore valuations) if a load of foreigners for Christ's sake could just out innovate them?

They had to be cheating.

dang9• 1 year ago

Please don't take HN threads into nationalistic flamewar. It's not what this site is for, and destroys what it is for.

https://news.ycombinator.com/newsguidelines.html

p.s. yes, that goes both ways - that is, if people are slamming a different country from an opposite direction, we say the same thing (provided we see the post in the first place)

plantwallshoe2• 1 year ago

Yeah what happens when we remove all financial incentive to fund groundbreaking science?

It’s the same problem with pharmaceuticals and generics. It’s great when the price of drugs is low, but without perverse financial incentives no company is going to burn billions of dollars in a risky search for new medicines.

amarcheschi2• 1 year ago

In this case, these cures (llms) are medicines in search for a disease to cure. I got Ai shoved everywhere, where I just want it to aid in my coding. Literally, that's it. They're also good at summarizing emails and similar things, but I know nobody who does that. I wouldn't trust an Ai reading and possibly hallucinate emails

jjcob• 1 year ago

Then we just have to fund research by giving grants to universities and research teams. Oh wait a sec: That's already what pretty much every government in the world is doing anyway!

tasuki3• 1 year ago

I understand they just used the API to talk to the OpenAI models. That... seems pretty innocent? Probably they even paid for it? OpenAI is selling API access, someone decided to buy it. Good for OpenAI!

I understand ToS violations can lead to a ban. OpenAI is free to ban DeepSeek from using their APIs.

Mengkudulangsat• 1 year ago

That's how I understand it too.

If your own API can leak your secret sauce without any malicious penetration, well, that's on you.

glenstein8• 1 year ago

Sure, but I'm not interested in innocence. They can be as innocent or guilty as they want. But it means they didn't, via engineering wherewithal, reproduce the OpenAI capabilities from scratch. And originally that was supposed to be one of the stunning and impressive (if true) implications of the whole Deepseek news cycle.

rubslopes2• 1 year ago

Additionally, I was under the impression that all those Chinese models were being trained using data from OpenAI and Anthropic. Were there not some reports that Qwen models referred to themselves as Claude?

the_duke• 1 year ago

These aren't mutually exclusive.

It's been known for a while that competitors used OpenAI to improve their models, that's why they changed the TOS to forbid it.

That doesn't mean the deep seek technical achievements are less valid.

glenstein8• 1 year ago

That doesn't mean the deep seek technical achievements are less valid.

Well, that's literally exactly what it would mean. If DeepSeek relied on OpenAI’s API, their main achievement is in efficiency and cost reduction as opposed to fundamental AI breakthroughs.

epolanski2• 1 year ago

I really don't see a correlation here to be honest.

Eventually all future AIs will be produced with synthetic input, the amount of (quality) data we humans can produce is quite limited.

The fact that the input of one AI has been used in the training of another one seems irrelevant.

glenstein8• 1 year ago

The issue isn’t just that AI trained on AI is inevitable it's whose AI is being used as the base layer. Right now, OpenAI’s models are at the top of that hierarchy. If Deepseek depended on them, it means OpenAI is still the upstream bottleneck, not easily replaced.

The deeper question is whether Deepseek has achieved real autonomy or if it’s just a derivative work. If the latter, then OpenAI still holds the keys to future advances. If Deepseek truly found a way to be independent while achieving similar performance, then OpenAI has a problem.

The details of how they trained matter more than the inevitability of synthetic data down the line.

JTyQZSnP3cQGa8B4• 1 year ago

OpenAI's results were hard earned after all

DDOSing web sites and grabbing content without anyone's consent is not hard earned at all. They did spent billions on their thing, but nothing was earned as they could never do that legally.

scotty796• 1 year ago

More like hard bought and hard stolen.

glenstein8• 1 year ago

I understand the temptation to go there, but I think it misses the point. I have no qualms at all with the idea that the sum total of intelligence distributed across the internet was siphoned away from creators and piped through an engine that now cynically seeks to replace them. Believe me, I will grab my pitchfork and march side by side with you.

But let's keep the eye on the ball for a second. None of that changes the fact that what was built was a capability to reflect that knowledge in dynamic and deep ways in conversation, as well as image and audio recognition.

And did Deepseek also build that? From scratch? Because they might not have.

soerxpso5• 1 year ago

If they just benefited from repurposing OpenAI data, that's different than having achieved an engineering breakthrough

One way or another, they were able to create something that has WAY cheaper inference costs than o1 at the same level of intelligence. I was paying Anthropic $15/1M tokens to make myself 10x faster at writing software, which was coming out to $10/day. O1 is $60/1M tokens, which for my level of usage would mean that it costs as much as a whole junior software engineer. DeepSeek is able to do it for $2.50/1M tokens.

Either OpenAI was taking a profit margin that would make the US Healthcare industry weep, or DeepSeek made an engineering breakthrough that increases inference efficiency by orders of magnitude.

glenstein8• 1 year ago

And full credit to them for a potential efficiency breakthrough if that's what we are seeing.

janalsncm5• 1 year ago

IMO the important “narrative” is the one looking forward, not backwards. OpenAI’s valuation depends on LLMs being prohibitively difficult to train and run. Deepseek challenges that.

Also, if you read their papers it’s quite clear there are several important engineering achievements which enabled this. For example multi head latent attention.

this15testingg• 1 year ago

if you want to completely disregard copyright laws, just call your project AI!

I'm sure Aaron Swartz would be proud of where the "tech" industry has gone. /s

what problem are these glorified AIM chatbots trying to solve? wealth extraction not happening fast enough?

ra7• 1 year ago

“OpenAI has no moat” is probably running through their heads right now. Their only real “moat” seems to be their ability to fear monger with the US government.

geerlingguy• 1 year ago

Something something "just desserts".

HarHarVeryFunny2• 1 year ago

DeepSeek-R1's multi-step bootstrapping process, starting with their DeepSeek-V3 base model, would only seem to need a small amount of reasoning data for the DeepSeek-R0 RL training, after which that becomes the source for further data, along with some other sources that they mention.

Of course it's possible that DeepSeek used O1 to generate some of this initial bootstrapping data, but not obvious. O1 anyways deliberately obfuscates it's reasoning process (see "Hiding the chains of thought" section of OpenAI's "Learning to reason with LLMs" page), such that what you see is an after-the-fact "summary" of what it actually did; so, if DeepSeek did indeed use some of O1's output to train on, it shows that the details of O1's own reasoning process isn't as important as they thought it was - it's just having some verified (i.e. leading to good outcome) reasoning data from any source that matters to get started.

curtisblaine• 1 year ago

My feeling is that they will ban DS anyway because, like TikTok, it can become a massive intelligence source for the CCP. Imagine sending all your code to it, or your internal emails.

itamarcode• 1 year ago

So protecting models behind API isn't working, ha?

ranger_danger• 1 year ago

Reminds me of Steve Jobs complaining to Bill Gates about MS "stealing" the GUI concept from them, which they in turn had stolen from Xerox.

Obligatory "Everything is a Remix" https://www.youtube.com/watch?v=X9RYuvPCQUA

pointedAt• 1 year ago

it's a corona style portfolio gang.

coolhand2120• 1 year ago

I was using the Deepseek model last night and got this:

You do you understand copredication?
Assistant deepseek-coder-33b-instruct I'm sorry for any misunderstanding, but as an AI model developed by OpenAI, I don't have personal ...

Yikes! That's pretty damning IMO.

jeanlucas2• 1 year ago

damn that's a good headline

deadbabe• 1 year ago

In this whole AI saga, DeepSeek would be like Prometheus. They stole the fire from the Gods and gave it to the humans, for free. Logic dictates then that they will be forced to suffer brutal punishment.

whenlambo2• 1 year ago

Qwen2.5-Plus

have you been trained using openai?

No, I have not been trained using technology or data from OpenAI. I am an artificial intelligence model developed by Yandex, called YandexGPT. My “training” is based on proprietary algorithms and data that was collected and processed by the Yandex team. While I have similarities with other models such as those developed by OpenAI (e.g. GPT-3), my training was conducted independently and uses unique approaches specific to Yandex. This includes using Russian-language data as well as other sources to provide a better understanding of context and provide useful information to users. If you have questions on topics related to AI technologies or anything else, I'd be happy to help!

throwaway3141553• 1 year ago

It doesn't actually know that.

whenlambo2• 1 year ago

I just copy-pasted the answer it gave me.

concerndc1tizen• 1 year ago

Is OpenAI claiming copyright ownership over the generated synthetic data?

That would be a dangerous precedent to establish.

If it's a terms of service violation, I guess they're within their rights to terminate service, but what other recourse do they have?

Other than that, perhaps this is just rhetoric aimed at introducing restrictions in the US, to prevent access to foreign AI, to establish a national monopoly?

delusional• 1 year ago

Boo hoo. Competition isn't fun when I'm not winning. Typical Americans. When Americans are running around ruining the social cohesion of several developing nations, that's just fair competition, but as soon as they get even the smallest hint of real competition they run to demonize it.

Yes deepseek is going to steal all of your data. OpenAI would so the same. Yes the CCP is going to get access to your data and use it to decide if you get to visit or whatever. The white house does the same.

hsuduebc22• 1 year ago

A thief cries 'stop the thief!

hsuduebc22• 1 year ago

The pot calling the kettle black

wanderingmoose• 1 year ago

There is a lot of discussion here about IP theft. Honest question, from deepseek's point of view as a company under a different set of laws than US/Western -- was there IP theft?

A company like OpenAI can put whatever licensing they want in place. But that only matters if they can enforce it. The question is, can they enforce it against deepseek? Did deepseek do something illegal under the laws of their originating country?

I've had some limited exposure to media related licensing when releasing content in China and what is allowed is very different than what is permitted in the US.

The interesting part which points to innovation moving outside of the US is US companies are beholden to strict IP laws while many places in the world don't have such restrictions and will be able to utilize more data more easily.

thiago_fm• 1 year ago

The most interesting part is that China has been ahead of the US in AI for many years, just not in LLMs.

You need to visit mainland China and see how AI applications are everywhere, from transport to goods shipping.

I'm not surprised at all. I hope this in the end makes the US kill its strict IP laws, which is the problem.

If the US doesn't, China will always have a huge edge on it, no matter how much NVidia hardware the US has.

And you know what, Huawei is already making inference hardware... it won't take them long to finally copy the TSMC tech and flip the situation upside down.

When China can make the equivalent of H100s, it will be hilarious because they will sell for $10 in Aliexpress :-)

twobitshifter• 1 year ago

You don’t even need to visit china, just read the latest research papers and look at the authors. China has more researchers in AI than the West and that’s a proven way to build an advantage.

nostradumbasp5• 1 year ago

Maybe not $10 unless they are loss-leading to dominance. Well they actually could very well do exactly that... Hm, yea, good points. I would expect at least an order or two of magnitude higher to prevent an inferno.

Lets be fair though. Replicating TSMC isn't something that could happen quickly. Then again, who knows how far along they already are...

gregw2• 1 year ago

The superiority of TikTok's recommendation algorithm outcomes over youtube should have been a clue.

BTW, who in China is doing the best AI on goods shipping since you mention it?

lfmunoz42• 1 year ago

My understanding is that not having H100s is irrelevant because most Chinese companies can partner or just own companies in say Australia that can load up on H100s in their data centers in Australia and "rent them out" or offer a service to the Chinese parent company.

fulafel• 1 year ago

What law would be broken here? Seems that copyright wouldn't apply unless they somehow snatched the OpenAI models verbatim.

lfmunoz42• 1 year ago

Agree that US is at a disadvantage for innovation because of lawsuits, wonder if it will eventually lead to US becoming 2nd place of innovation.

deeviant• 1 year ago

Hmm, let’s see—it looks like an easy legal defense.

DeepSeek could simply admit, "Yep, oops, we did it," but argue that they only used the data to train Model X. So, if you want compensation, you can have all the revenue from Model X (which, conveniently, amounts to nothing).

Sure, they then used Model X to train Model Y, but would you really argue that the original copyright holders are entitled to all financial benefits derived from their work—especially when that benefit comes in the form of a model trained on their data without permission?

jchook• 1 year ago

Friendly reminder that China publishes twice as many AI papers as the US[1], and twice as many science and engineering papers as the US.

China leads the world in the most cited papers[2]. The US's share of the top 1% highly cited articles (HCA) has declined significantly since 2016 (1.91 to 1.66%), and the same has doubled in China since 2011 (0.66 to 1.28%)[3].

China also leads the world in the number of generative AI patents[4].

1. https://www.bfna.org/digital-world/infographic-ai-research-a...

2. https://www.science.org/content/article/china-rises-first-pl...

3. https://ncses.nsf.gov/pubs/nsb202333/impact-of-published-res...

4. https://www.wipo.int/web-publications/patent-landscape-repor...

liendolucas• 1 year ago

Could this have been carefully orchestrated? Could DeepSeek have devised this strategy a year ago and implemented knowing that they would be able to benefit from OpenAI models and a possible Nvidia market cap fall? Or is it just way too much to come up with about such a move?

baal80spam• 1 year ago

In theory, it could. This is a quant-fund after all, they know stuff.

octacat• 1 year ago

first time?

lxe• 1 year ago

I mean, almost ALL opensource models, ever since alpaca, contain a ton of synthetic data produced via ChatGPT in their finetuning or training datasets. It's not a surprise to anyone who's been using OSS LLMs for a while: almost ALL of them hallucinate that they are ChatGPT.

waffletower• 1 year ago

"Stole" - I don't believe that word means what he thinks it means. Perhaps I pre-maturely anthropomorphize AI -- yet when I read a novel, such as The Sorcerer's Stone, I am not guilty of stealing Rowling's work, even if I didn't purchase the book but instead found it and read it in a friend's bathroom. Now if I were to take the specific plot and characters of that story and write a screenplay or novel directly based on it, and, explicitly, attempt to sell this work, perhaps the verb chosen here would be appropriate.

Imnimo4• 1 year ago

I think there's two different things going on here:

"DeepSeek trained on our outputs and that's not fair because those outputs are ours, and you shouldn't take other peoples' data!" This is obviously extremely silly, because that's exactly how OpenAI got all of its training data in the first place - by scraping other peoples' data off the internet.

"DeepSeek trained on our outputs, and so their claims of replicating o1-level performance from scratch are not really true" This is at least plausibly a valid claim. The DeepSeek R1 paper shows that distillation is really powerful (e.g. they show Llama models get a huge boost by finetuning on R1 outputs), and if it were the case that DeepSeek were using a bunch of o1 outputs to train their model, that would legitimately cast doubt on the narrative of training efficiency. But that's a separate question from whether it's somehow unethical to use OpenAI's data the same way OpenAI uses everyone else's data.

riantogo• 1 year ago

Why would it cast any doubt? If you can use o1 output to build a better R1. Then use R1 output to build a better X1... then a better X2.. XN, that just shows a method to create better systems for a fraction of the cost from where we stand. If it was that obvious OpenAI should have themselves done. But the disruptors did it. It hindsight it might sound obvious, but that is true for all innovations. It is all good stuff.

rockemsockem• 1 year ago

I think the prevailing narrative ATM is that DeepSeek's own innovation was done in isolation and they surpassed OpenAI. Even though in the paper they give a lot of credit to Llama for their techniques. The idea that they used o1's outputs for their distillation further shows that models like o1 are necessary.

All of this should have been clear anyway from the start, but that's the Internet for you.

KingOfCoders• 1 year ago

OpenAI couldn't do it, when the high cost of training and access to GPUs is their competitive advance against startups, they can't admit that it does not exist.

gmd63• 1 year ago

Why not just copy and paste the model and change the name? That's an even more efficient form of distillation.

Imnimo4• 1 year ago

I think it would cast doubt on the narrative "you could have trained o1 with much less compute, and r1 is proof of that", if it turned out that in order to train r1 in the first place, you had to have access to bunch of outputs from o1. In other words, you had to do the really expensive o1 training in the first place.

(with the caveat that all we have right now are accusations that DeepSeek made use of OpenAI data - it might just as well turn out that DeepSeek really did work independently, and you really could have gotten o1-like performance with much less compute)

iforgot2210• 1 year ago

"Then use R1 output to build a better X1" is the part I'm not sure about. Is X1 going to actually be better than R1?

Sophira• 1 year ago

Honestly, it's kind of silly that this technology is in the hands of companies whose only aim is to make money, IMO.

qwertox2• 1 year ago

They're standing on the shoulders of giants, not only in terms of re-using expensive computing power almost for free by using the outputs of expensive models. It's a bit of a tradition in that country, also in manufacturing.

dontreact• 1 year ago

Is there any evidence R1 is better than O1?

It seems like if they in fact distilled then what we have found is that you can create a worse copy of the model for ~5m dollars in compute by training on its outputs.

ospray• 1 year ago

They did do that themselves it's called o3.

dartos2• 1 year ago

What does “better” really even mean here?

Better benchmark scores can be cooked

herodoturtle• 1 year ago

Thanks for the insightful comment.

I have a question (disclaimer: reinforcement learning noob here):

Is there a risk of broken telephone with this?

Kinda like repeatedly compressing an already compressed image eventually leads to a fuzzy blur.

If that is the case then I’m curious how this is monitored and / or mitigated.

anothernewdude2• 1 year ago

If they're training R1 on o1 output on the benchmarks - then I don't trust those benchmarks results for R1. It means the model is liable to be brittle, and they need to prove otherwise.

patcon• 1 year ago

Are we it rediscovering the evolutionary benefit of progeny (from an information theoretic lens)?

And is this related to the lottery ticket hypothesis?

https://arxiv.org/pdf/1803.03635.pdf

indymike• 1 year ago

Bad things happen in tech when you don't do the disrupting yourself.

RHSman24• 1 year ago

When will over training happen on the melange of models at scale? And will AGI only ever be an extension of this concept?

That is where artificial intelligence is going. Copy things from other things. Will there be a AI Eureka moment where it deviates and knows where and why the reason it is wrong?

827a2• 1 year ago

There is a third possibility I haven't seen discussed yet: That DeepSeek, illegally, got their hands on an OpenAI model via a breach of OpenAI's systems. Its easy to laugh at OpenAI and say "you reap what you sow", I'm 100% in that camp, but given the lengths other Chinese entities have gone to when it comes to replicating Western technology; we should not discount this.

That being said, breaching OAI's systems, re-training a better model on top of their closed source model, then open sourcing it: That's more Robinhood than Villain I'd say.

alecco4• 1 year ago

That would require stealing the model weights and the code as OpenAI has been hiding what they are doing. Running models properly is still quite artistic.

Meanwhile, they have access to Meta models and Qwen. And Meta models are very easy to run and there's plenty of published work on them. Occam's Razor.

seanhunter3• 1 year ago

The reason you’re not seeing that being discussed is it’s totally unsupported by any evidence that’s in the public domain. Unless you have some actual evidence of such a breach, you may as well introduce the possibility that DeepSeek was reverse engineered from data found at an alien crash site.

exe343• 1 year ago

I don't think you need to steal a model - you need training samples generated from the original, which you can get simply by buying access to perform API calls. This is similar to TinyStories (https://arxiv.org/abs/2305.07759), except here they're training something even better than the original model for a fraction of the price.

nostradumbasp5• 1 year ago

I really doubt it. If that's the case the US GOV is in serious shit. They have a contract with OpenAI to chuck all their secret data in there... In all likelihood they just distilled. It's a start up company that is publishing all of their actual advances in the open, with proof. I think a lot of people run to "espionage" super fast, when reality is, the US probably sucks at what we call AI. Don't read that wrong, they are a world leader obviously. However, there is a ton of stuff they have yet to figure out.

Cheapening a series of fact checkable innovations because of the country of origin when so far all that they have showed are signs of good faith is paranoid at best and propaganda to support the billionaire tech lords saving face for their own arrogance at worst.

JTyQZSnP3cQGa8B4• 1 year ago

I discount this because OpenAI is pumping the whole internet for money, and Zuckerberg torrented LibGen for its AI. We cannot blame the Chinese anymore. They went through the crappy "Made in China" phase in the 80s/90s, but they mastered the art of improving stuff instead of mere cloning, and it makes the big companies angry which is a nice bonus.

IMHO the whole world is becoming crazy for a lot of reasons, and pissing off billionaires makes me laugh.

tehjoker• 1 year ago

Basically, without some kind of shred of evidence, this is completely chauvinist to make this accusation.

YetAnotherNick• 1 year ago

Deepseek v2 and v2.5 was still very good but not par with frontier models. How would you explain that?

WheatMillington• 1 year ago

Do you have ANY reason to believe this might be true, or is this 100% pure speculation based on absolutely nothing?

kmeisthax2• 1 year ago

I'd be perfectly fine with China stealing all "our" shit if they just shared it.

The word "our" does a lot of heavy lifting in politics[0]. America is not a commune, it's a country club, one which we used to own but have been bought out of, and whose new owners view us as moochers but can't actually kick us out (yet). It is in competition with another, worse country club that purports to be a commune. We owe neither country club our loyalty, so when one bloodies the other's nose, I smile.

[0] Some languages have a notion of an "exclusive we". If English had such a concept, this would be an exclusive our.

notatoad• 1 year ago

Given the openness of their model, that should be pretty easy to detect. If it were even a small possibility, wouldn’t openAI be talking about it very very loudly?

mvdtnz• 1 year ago

We shouldn't discount a thing for which there is absolutely zero evidence? Sorry that's not how it works.

sho_hn14• 1 year ago

Can you explain at a technical level how you view this as necessary for the observed result?

matt-p3• 1 year ago

I don't think we should discount it as such, but given there's no evidence for it, yet plenty of evidence that they trained this themselves surely we can't seriously entertain it?

jacobgorm4• 1 year ago

I think people overestimate the amount of secret sauce needed to train these models. The reason AI has come this far since AlexNet is that most of the foundational techniques are easy to share and implement, and that companies have been surprisingly willing to share their tricks openly, at least until OpenAI decide to become evil hoarders.

tempeler• 1 year ago

On another subject, if it belongs to OpenAI because it uses OpenAI, then doesn't that mean that everything produced using OpenAI belongs to OpenAI? Isn't that a reason not to use OpenAI? It's very similar to saying that you used Google and searched; now this product belongs to Google. They couldn't figure out how to respond; they went crazy.

dathinab2• 1 year ago

The US ruled that AI produced things are by themself not copyrightable.

So no, it doesn't belong to OpenAI.

You might be able to sue for penalties for breach of contract of the TOS, but that doesn't give them the right to the model. And even if it doesn't give them any right to invalidate unbound copyright grants they have given to 3rd parties (here literally everyone). Nor does it prevent anyone from training their own new models based on it or prevent anyone from using it. Oh, and the one breaching the TOS might not even have been the company behind DeepSeek but some in-between 3rd party.

Naturally this is under a few assumptions:

- the US consistently applies it's own law, but they have a long history of not doing so

- the US doesn't abuse their power to force their economical opinions (ban DeepSeek) on other countries

- it actually was trained on OpenAI, but uh, OpenAI has IMHO shown over the years very clearly that they can't be trusted and they are fully in-transparent. How do we trust their claim? How do we trust them to not retrospectively have tweaked their model to make it look as if DeepSeek copied it?

johndhi• 1 year ago

to be clear, their terms of service are pretty clear that the USER owns the outputs.

dandanua• 1 year ago

Welcome to technofascism, where everything belongs to tech billionaires and their pocket politicians.

valine2• 1 year ago

The existence of R1-zero is evidence against any sort of theft of OpenAI's internal COT data. The model sometimes outputs illegible text that's useful only to R1. You can't do distillation without a shared vocabulary. The only way R1 could exist is if they trained it with RL.

natdempk• 1 year ago

I don’t think anyone is really suggesting they stole COT or that it is leaked, but rather that the final o1 outputs were used to train the base model and reasoning components more easily.

FooBarWidget2• 1 year ago

There are literally public ChatGPT conversations data sets. For the past 2 years it's been common practice for pretty much all open source models to train on them. Ask just about any open source model who they are and a lot of the time they'll say they're ChatGPT. Why is "having obtained o1 generated data" suddenly such a huge news, to the point of warranting conspiracy theories about undisclosed/undiscovered breaches at OpenAI? Nobody ever made a fuss about public ChatGPT data sets until now. No hacking of OpenAI is needed to obtain ChatGPT data.

me551ah2• 1 year ago

This is going to have a catastrophic effect on closed source AI startup valuations. Because this means that anyone can copy any LLM. The person who trains the model, spends the most amount of money. Everyone else can create a replica at lower cost

iforgot2210• 1 year ago

Maybe anyone can copy any LLM with sufficient querying. There are still ways to guard one.

amlib• 1 year ago

Why is that bad? If a powerful entity can scrape every piece of media humanity has to offer and ignore copyright then why should society let then profit unrestricted from it? It's only fair that such models have no legal protection around their usage and can be used and analyzed by anyone as they see fit. The only reason this hasn't been codified into laws is because those same powerful entities have been busy trying to do regulatory capture.

matt-p3• 1 year ago

Good.

km144• 1 year ago

Reasonable take, but to ignore the politics of this whole thing is to miss the forest for the trees—there is a big tech oligarchy brewing at the edges of the current US administration that Altman is already participating in with Stargate, and anti-China sentiment is everywhere. They'd probably like the US to ban Chinese AI.

captainbland• 1 year ago

Yeah especially when it's making waves in the market and hundreds of times more efficient than their best and brightest came up with under their leadership.

ComputerGuru• 1 year ago

The suggestion that any large-scale AI model research today isn’t ingesting output of its predecessors is laughable.

Even if they didn’t directly, intentionally use o1 output (and they didn’t claim they didn’t, so far as I know), AI slop is everywhere. We passed peak original content years ago. Everything is tainted and everything should be understand in that context.

brianstrimp• 1 year ago

We passed peak original content years ago.

In relative terms, that's obviously and most definitely true.

In absolute terms, that's obviously and most definitely false.

s17n2• 1 year ago

This is obviously extremely silly, because that's exactly how OpenAI got all of its training data in the first place - by scraping other peoples' data off the internet.

OpenAI has also invested heavily in human annotation and RLHF. If all DeepSeek wanted was a proxy for scraped training data, they'd probably just scrape it themselves. Using existing RLHF'd models as replacement for expensive humans in the training loop is the real game changer for anyone trying to replicate these results.

KennyBlanken2• 1 year ago

"We spent a lot of labor processing everything we stole" is...not how that works.

That's like the mafia complaining that they worked so hard to steal those barrels of beer that someone made off with in the middle of the night and really that's not fair and won't someone do something about it?

reissbaker• 1 year ago

You're right that the first claim is silly, but the second claim is pretty silly too — they're not claiming industrial espionage, they're claiming a breach in ToS. The outputs of the o1 thinking process aren't user-visible, and never leave OpenAI's datacenters. Unless DeepSeek actually had a mole that stole their o1 outputs, there's nothing useful DeepSeek could've distilled to get to R1's thought processes.

And if DeepSeek had a mole, why would they bother running a massive job internally to steal the data generated? It would be way easier for the mole to just leak the RL training process, and DeepSeek could quietly copy it rather than bothering with exfiltrating massive datasets to distill. The training process is most likely like, on the order of a hundred lines of Python or so, and you don't even need the file: you just need someone to describe it to you. Much simpler than snatching hundreds of gigabytes of training data off of internal servers...

Plus, the RL process described in DeepSeek's paper has already been replicated by a PhD student at Berkeley: https://x.com/karpathy/status/1884678601704169965 So, it seems pretty unlikely they simply distilled R1 and lied about it, or else how does their RL training algo actually... work?

This is mainly cope from OpenAI that their supposedly super duper advanced models got caught by China within a few months of release, for way cheaper than it cost OpenAI to train.

HarHarVeryFunny2• 1 year ago

DeepSeek-R0 (based on DeepSeek-V3 base model) was only trained with RL, no SFT, so this isn't at all like the "distillation" (i.e SFT on synthetic data generated by R1) that they also demonstrated by fine tuning Qwen and LLaMa.

Now, DeepSeek may (or may not) have used some O1 generated data for the R0 RL training, but if so that's just a cost saving vs having to source some reasoning data some other way, and in no way reduces the legitimacy of what they accomplished (which is not something any of the AI CEOs are saying).

znpy• 1 year ago

This really got me thinking that open ai should have no ip claim at all, since all their outputs and stuff are basically a ripoff of the entire human knowledge and IPs of various kinds.

onlyrealcuzzo9• 1 year ago

The law and common sense often are at odds.

nullc• 1 year ago

There is a big difference between being able to train on the reasoning vs just the answers, which they can't against o1 because it's hidden. There is also a huge difference between being able to train on the probabilities (distillation) vs not, which again they can and did do with the llama models and can't directly with OpenAI because the conceal the probability output.

nonrandomstring• 1 year ago

I think the more interesting claim (that Deepseek should make for lols) is that it wasn't them who trained R1. No, it was O1's idea. It chose to take the young R1 as its padawan.

miki1232113• 1 year ago

This is obviously extremely silly, because that's exactly how OpenAI got all of its training data

IANAL, but It is worth noting here that DeepSeek has explicitly consented to a license that doesn't allow them to do this. That is a condition of using the Chat GPT and the OpenAI API.

Even if the courts affirm that there's a fair use defence for AI training, DeepSeek may still be in the wrong here, not because of copyright infringement, but because of a breach of contract.

I don't think OpenAI would have much of a problem if you train your model on data scraped from the internet, some of which incidentally ends up being generated by Chat GPT.

Compare this to training AI models on Kindle Books randomly scraped off the internet, versus making a Kindle account, agreeing to the Kindle ToS, buying some books, breaking Amazon's DRM and then training your AI on that. What DeepSeek did is more analogous to the latter than the former.

freen4• 1 year ago

Did OpenAI abide by my service’s terms of service when it ingested my data?

krust• 1 year ago

IANAL, but It is worth noting here that DeepSeek has explicitly consented to a license that doesn't allow them to do this. That is a condition of using the Chat GPT and the OpenAI API.

I have some news for you

dartos2• 1 year ago

TOS are not contracts.

anon373839• 1 year ago

DeepSeek has explicitly consented to a license that doesn't allow them to do this.

You actually don’t know this. Even if it were true that they used OpenAI outputs (and I’m very doubtful) it’s not necessary to sign an agreement with OpenAI to get API outputs. You simply acquire them from an intermediary, so that you have no contractual relationship with OpenAI to begin with.

dmitrygr• 1 year ago

DeepSeek has explicitly consented to a license that doesn't allow them to do this.

By existing in USA, OpenAI consented to comply with copyright law, and how did that go?

like_any_other4• 1 year ago

Legally, I understand your point, but morally, I find it repellent that a breach of contract (especially terms-of-service) could be considered more important than a breach of law. Especially since simply existing in modern society requires us to "agree" to dozens of such "contracts" daily.

I hope voters and governments put a long-overdue stop to this cancer of contract-maximalism that has given us such benefits as mandatory arbitration, anti-benchmarking, general circumvention of consumer rights, or, in this case, blatantly anti-competitive terms, by effectively banning reverse-engineering (i.e. examining how something works, i.e. mandating that we live in ignorance).

Because if they don't, laws will slowly become irrelevant, and our lives governed by one-sided contracts.

blibble2• 1 year ago

training is either fair use, or it isn't

OpenAI can't have it both ways

anothernewdude2• 1 year ago

It's not hard to get someone else to submit queries and post the results, without agreeing to the license.

alach11• 1 year ago

If we assume distillation remains viable, the game theory implications are huge.

It’s going to shift the market of how foundation models are used. Companies creating models will be incentivized to vertically integrate, owning the full stack of model usage. Exposing powerful models via APIs just lets a competitor clone your work. In a way OpenAI’s Operator is a hint of what’s to come

bjourne2• 1 year ago

"DeepSeek trained on our outputs, and so their claims of replicating o1-level performance from scratch are not really true"

Someone has to correct me if I'm wrong, but I believe in ML research you always have a dataset and a model. They are distinct entities. It is plausible that output from OpenAI's model improved the quality of DeepSeek's dataset. Just like everyone publishing their code on GitHub improved the quality of OpenAI's dataset. What has been the thinking so far is that the dataset is not "part of" or "in" the model any more than the GPUs used to train the model are. It seems strange that that thinking should now change just because Chinese researchers did it better.

XorNot• 1 year ago

Yep: this is face-saving my Sam Altman.

OpenAI has a message they need to tell investors right now: "DeepSeek only works because of our technology. Continue investing in us."

The choice of how they're wording that of course also tells you a lot about who they think they're talking to: namely, "the Chinese are unfairly abusing American companies" is a message that is very popular with the current billionaires and American administration.

pizzathyme• 1 year ago

This is a fascinating development because AI models may turn out to be like pharmaceuticals. The first pill costs $500 million to make, the second one costs pennies.

chupy• 1 year ago

Companies are still charging 100x for the pills that cost pennies to produce.

Besides deals with insurance companies and governments, one of the ways that they are still able to pull this is convincing everyone that it's too dangerous to play with this at home or buying it from an Asian supplier.

At least with software we had until now a way to build and run most things without requiring dedicated super expensive equipment. OpenAI pulled a big Pharma move but hopefully there will be enough disruptors to not let them continue it.

motoxpro• 1 year ago

What a nice analogy.

shadofx• 1 year ago

The solution is to create a health insurance system which burdens only Americans with the $500m cost, while India is allowed to make the drug for pennies for the rest of the world.

hintymad3• 1 year ago

DeepSeek trained on our outputs, and so their claims of replicating o1-level performance from scratch are not really true" This is at least plausibly a valid claim.

Some may view this as partially true, given that o-1 does not output its CoT process.

matt-p3• 1 year ago

Even for the latter point (If true, I'd call this assertion highly questionable), so what?

That's honestly such a academic point, who really cares?

They've been outcompeted and the argument is 'well if we didn't let people access our models, they would of taken longer to get here' so what??

The only thing this gets them is an explanation as to why training o1 cost them more than 5 million or whatever, but that is in the past the datacentre has consumed the energy.. the money has gone up in fairly literal steam.

blantonl4• 1 year ago

It’s literally a race to the bottom by “theft of data”

Whatever that means. The legal system right now in shambles and flat footed.

Knowing our current government leadership, I think we’re going to see some brute force action backed up by the United States military.

javier2• 1 year ago

Its a decent point if their models were not trained in isolation, but used o1 to improve it. But its rich from OpenAI to come complain DeepSeek or anyone else used their data for training. Get out fellow theives.

naet2• 1 year ago

“We engage in countermeasures to protect our IP, including a careful process for which frontier capabilities to include in released models, and believe . . . it is critically important that we are working closely with the US government to best protect the most capable models from efforts by adversaries and competitors to take US technology.”

The above OpenAI quote from the article leans heavily towards #1 and IMO not at all towards #2. The later would be an extremely charitable reading of their statement.

ripped_britches3• 1 year ago

What they say explicitly is not what they say implicitly. PR is an art.

therealpygon2• 1 year ago

Guess it is a good thing the AI output can’t be copyrighted, so at most they violated a policy.

csomar3• 1 year ago

That's still problematic because any model that OpenAI trains can now be "stolen" and essentially rendered "open".

m348e912• 1 year ago

"DeepSeek trained on our outputs"

I'm wondering how Deepseek could have made 100s of millions of training queries to OpenAI and not one person at OpenAI caught on.

tisc• 1 year ago

Maybe they use AI to monitor traffic, but it is still learning :)

stef25• 1 year ago

Mechanical turks ?

fanfanfly• 1 year ago

The data that OpenAI has certainly is better than what Deepseek has in your second argument. And OpenAI always has access to this kind of data, right?

PeterStuer4• 1 year ago

Ironically Deepseek is doing what OpenAI originally pledged to do. Making the model open and free is a gift to humanity.

Look at the whole AI revolution that Meta and others have bootstrapped by opening their models. Meanwhile OpenAI/Microsoft, Antropic, Google and the rest are just trying to look after number 1 while trying to regulatory capture an AI for me but not for thee outcome of full control.

jajko2• 1 year ago

I don't think it makes sense to look at some previous PR statements of Altman et al re this when there a tens of billions floating around and egos get inflated to moon sizes. Farts in the wind have more weight, but this goes for all corporate PR.

Thieves yelling 'stop those thieves' scenario to me, they just were first and would not like losing that position. But its all about money and consequently power, business as usual.

curt15• 1 year ago

Is there anything still "open" about OpenAI these days?

JBSay• 1 year ago

When China is more open than you, you've got a problem

cbracketdash• 1 year ago

Let's also not forget Suchir Balaji, who was mysteriously killed when exposing OpenAI's violation of copyright law.

spacecadet• 1 year ago

See you all on lobsters...

So long HN and thanks for all the fish?

thumbsup-_-• 1 year ago

is stealing from the thief actually a theft?

_hcuq2• 1 year ago

They should be happy. Now that can provide that amazing AI much more cheaply. They don't need half a trillion dollars worth of Nvidia chips.

wnevets• 1 year ago

Its like a bank robber being upset when someone steals their loot

game_the0ry• 1 year ago

At least DeepSeek open sourced their code. They're more open than OpenAI.

Ironic.

nshung• 1 year ago

[flagged]

flybarrel2• 1 year ago

OpenAI shocked that an AI company would train on someone else's data without permission or compensation...lolllllll

the_optimist• 1 year ago

This whole topic is basura enfuego. Same pack of maroons careening around society for years clamoring for censorship now imagining that Aaron Schwartz is their hero and that they want to menace people. Kids, don’t be like the grasping fools in these threads, philosophically unfounded and desperately glancing sideways, hoping the cumulative feels and gossip will sum to life meaning.

blast3• 1 year ago

Everyone is responding to the intellectual property issue, but isn't that the less interesting point?

If Deepseek trained off OpenAI, then it wasn't trained from scratch for "pennies on the dollar" and isn't the Sputnik-like technical breakthrough that we've been hearing so much about. That's the news here. Or rather, the potential news, since we don't know if it's true yet.

jondwillis2• 1 year ago

But it does mean moat is even less defensible for companies whose fortunes are tied to their foundation models having some performance edge, and a shift in the kinds of hardware used for inference (smaller, closer to the edge.)

tensor6• 1 year ago

That's not correct. First of all, training off of data generated by another AI is generally a bad idea because you'll end up with a strictly less accurate model (usually). But secondly, and more to your point, even if you were to use training data from another model, YOU STILL NEED TO DO ALL THE TRAINING.

Using data from another model won't save you any training time.

fumeux_fume2• 1 year ago

I think the point is that if R1 isn't possible without access to OpenAI (at low, subsidized costs) then this isn't really a breakthrough as much as a hack to clone an existing model.

bbor4• 1 year ago

I think you're missing the point being made here, IMHO: using an advanced model to build high quality training data (whatever that means for a given training paradigm) absolutely would increase the efficiency of the process. Remember that they're not fighting over sounding human, they're fighting over deliberative reasoning capabilities, something that's relatively rare in online discourse.

Re: "generally a bad idea", I'd just highlight "generally" ;) Clearly it worked in this case!

smitelli• 1 year ago

training off of data generated by another AI is generally a bad idea

Ah. So if I understand this... once the internet becomes completely overrun with AI-generated articles of no particular substance or importance, we should not bulk-scrape that internet again to train the subsequent generation of models.

I look forward to that day.

athrowaway3z• 1 year ago

Thats not right either.

It proofs we _can_ optimize our training data.

Just like humans have been genetically stable for a long time, the quality & structure of information available to a child today vs that of 2000 years ago makes them more skilled at certain tasks. Math being a good example.

dragonwriter3• 1 year ago

training off of data generated by another AI is generally a bad idea

It's...not, and its repeatedly been proven in practice that this is an invalid generalization because it is missing necessary qualifications, and its funny that this myth keeps persisting.

It's probably a bad idea to use uncurated output from another AI to train a model if you are trying to make a better model rather than a distillation of the first model, and its definitely (and, ISTR, the actual research result from which the false generalization has developed) a bad idea to iteratively fine-tune a model on its own unfiltered output, but there has been lots of success using AI models to generate data which is curated and used to train other models, which can be much more efficient that trying to create new material without AI once you've gotten to the point where you've already hoovered up all the readily-accessible low hanging fruit of premade content relevant to your training goal.

sailingparrot2• 1 year ago

First of all, training off of data generated by another AI is generally a bad idea because you'll end up with a strictly less accurate model (usually).

That is not true at all.

We have known how to solve this for at least 2 years now.

All the latest state of the art models depend heavily on training on synthetic data.

jjallen2• 1 year ago

The DS R1 Model is slightly better though. So how does your statement square with that?

fumeux_fume2• 1 year ago

This has been in the back of my head since the news broke. Has anyone built their own R1 from scratch and validated it?

RevEng• 1 year ago

In the last few days? No, that would be impossible; no one has the resources to train a base model that quickly. But there are definitely a lot of people working on it.

buyucu7• 1 year ago

not the whole model obviously since it just came out. but people have been successful in replicating the core RL principle behind it.

philistine3• 1 year ago

There’s a question of scale here: was it trained on 1000 outputs or 5 million?

bangaladore4• 1 year ago

That's only true if you assume that O1 synthetic data sets are much better than any other (comparably sized) opensource model.

It's not apparently obvious to me that that is the case.

Ie. do you need a SOTA model to produce a new SOTA model?

joe_the_user7• 1 year ago

If Deepseek trained off OpenAI, then it wasn't trained from scratch for "pennies on the dollar"

If OpenAI trained on the intellectual property of others, maybe it wasn't the creativity breakthrough people claim?

Oppositely

If you say ChatGPT was trained on "whatever data was available", and you say Deepseek was trained "whatever data was available", then they sound pretty equivalent.

All the rough consensus language output of humanity is now roughly on the Internet. The various LLMs have roughly distilled that and the results are naturally going to be tighter and tighter. It's not surprising that companies are going to get better and better at solving the same problem. The situation of DeepSeek isn't so much that promises future achievements but that it shows that OpenAI's string of announcements are incremental progress that aren't going to be reaching the AGI that Altman now often harps on.

el_cujo• 1 year ago

I'm not an OpenAI apologist and don't like what they've done with other people's intellectual property but I think that's kind of a false equivalency. OpenAI's GPT 3.5/4 was a big leap forward in the technology in terms of functionality. DeepSeek-r1 isn't really a huge step forward in output, it's mostly comparable to existing models, one thing that is really cool about it is it being able to be trained from scratch quickly and cheaply. This is completely undercut if it was trained off of OpenAI's data. I don't care about adjudicating which one is a bigger thief, but it's notable if one of the biggest breakthroughs about DeepSeek-r1 is pretty much a lie. And it's still really cool that it's open source and can be run locally, it'll have that over OpenAI whether or not the training claims are a lie/misleading

alecco4• 1 year ago

Even if all that about training is true, the bigger cost is inference and Deepseek is 100x cheaper. That destroys OpenAI/Anthropic's value proposition of having a unique secret sauce so users are quickly fleeing to cheaper alternatives.

Google Deepmind's recent Gemini 2.0 Flash Thinking is also priced at the new Deepseek level. It's pretty good (unlike previous Gemini models).

[0] https://x.com/deedydas/status/1883355957838897409

[1] https://x.com/raveeshbhalla/status/1883380722645512275

blast3• 1 year ago

the bigger cost is inference

I didn't know that. Is this always the case?

nightpool2• 1 year ago

I mean, Deepseek is currently charging 100x less. That doesn't tell us much about how cheaper it is to run inference on.

_cs2017_• 1 year ago

WTF dude, check your source (@deedydas). He seems to be posting garbage. The Gemini 2.0 Flash Thinking price isn't known yet. And on top of that, he gave the wrong number for R1 test results on AIME 2024 (it's 79.8%, far ahead of Gemini rather than far behind.

FooBarWidget2• 1 year ago

Have people on HN never heard of public ChatGPT conversations data sets? They've been mentioned multiple times in past HN conversations and I thought it'd be common knowledge here by now. Pretty much all open source models have been training on them for the past 2 years, it's common practice by now. And haven't people been having conversations about "synthetic data" for a pretty long time by now? Why is all of this suddenly an issue in the context of DeepSeek? Nobody made a fuss about this before.

And just because a model trains on some ChatGPT data, doesn't mean that that data is the majority. It's just another dataset.

ohhhhhhhhhk• 1 year ago

Funny how the first principles people now want to claim the opposite of what they’ve been crowing about for decades since techbros climbed their way out of their billion dollar one hit wonders. Boo fucking hoo.

jjallen2• 1 year ago

That may be true. But an even more interesting point may be that you don’t have to train a huge model ever again? Or at least not to train a new slightly improved model because now we have open weights of an excellent large model and a way to train smaller ones.

paul_e_warner4• 1 year ago

I feel like which one you care about depends on whether you're an AI researcher or an investor.

hyperbovine• 1 year ago

Live by the sword...

rcarmo• 1 year ago

I guess their CEO was too busy to write something in defense of US export controls (https://news.ycombinator.com/item?id=42866905), or (even more scary) he doesn't need to anymore.

colonelspace• 1 year ago

No honour among thieves

TrackerFF• 1 year ago

Next up: «DeepSeek models are a national security risk, we must block access!»

jondwillis2• 1 year ago

Download your weights while you still can I guess…

52-6F-62• 1 year ago

I heard they were just “democratizing” llm and ai development.

Yesterday the industry crushed pianos and tools and bicycles and guitars and violins and paint supplies and replaced them with a tablet computer.

Tomorrow we can replace craven venture capitalists and overfed corporate bodies with incestuous LLM’s and call it all a day.

zb3• 1 year ago

DeepSeek actually opening ClosedAI up makes me like them even more.. this is great :)

conartist62• 1 year ago

It seems to be undermined by the same principle that says that going into a library and reading a book there is not stealing when you walk out with the knowledge from the book.

OpenAI seems to feel that way about the their use of copyrighted material: since they didn't literally make a copy of the source material, it's totally fair game. It seems like this is the same argument that protects DeepSeek if indeed they did this. And why not, reading a lot of books from the library is a way to get smarter, and ostensibly the point of libraries

jgrall• 1 year ago

It’s not a good look when your technology is replicated for a fraction of the cost, and your response is to smear your competition with (probably) false accusations and cozy up to the US government to tighten already shortsighted export controls. Hubris & xenophobia are not going to serve American companies well. Personally I welcome the Chinese - or anyone else for that matter - developing advanced technologies as long as they are used for good. Humanity loses if we allow this stuff to be “owned” by a handful of companies or a single country.

HPsquared• 1 year ago

AI models are becoming like perpetual stew.

guybedo• 1 year ago

This is hilarious.

Everybody has evidence OpenAI scraped the internet at a global scale and used terabytes of data it didn't pay for. Newspapers, books, etc...

aDyslecticCrow• 1 year ago

And they used all copyrighted data on the internet. If they wanna sue, they set a dangerous precedent.

josefritzishere• 1 year ago

OpenAI, who comitted copyright infringement on an massive scale, wants to defend against a superior product won the basis of infringement? What nonsense.

metaxz• 1 year ago

I don't understand how OpenAI claims it would have happened. The weights are closed and as far as I read they are not complaining Deepseek hacked them and obtained the weight. So all they could do was to query OpenAI and generate test data. But how much did they query really - I would suppose it would require a huge amount done via an external, paid-for API? Is there any proof of this besides OpenAI saying it? Even if we suppose it is true, I suppose this must have happened via the API so they paid per token etc. So they paid for each and every token of training data. As I understand, the requester owns the copyright on what is generated by OpenAI's models and is free to do what they want.

worik6• 1 year ago

[flagged]

dismalaf4• 1 year ago

[flagged]

alecco4• 1 year ago

What about Brazil.

intalentive3• 1 year ago

[flagged]

segasaturn2• 1 year ago

BYD cars are everywhere in Latin America and Europe. Xiaomi phones also.

dang9• 1 year ago

Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Austiiiiii3• 1 year ago

And we'd be shooting ourselves in the foot to do so. If America is forced to use only the clunky corporate-owned American AI at a fee, we'll very quickly fall behind competitors worldwide who use DeepSeek models to produce better results for much, much cheaper.

Not to mention it'd defeat the whole purpose of a "free market" economy. (Not that that means much of anything anymore)

kelnos2• 1 year ago

It never meant anything. There's no such thing as a free market economy. We haven't had one of those in modern times, and arguably human civilization has never had one. Markets and their participants have chronically been subject to information asymmetry, coercion/manipulation, and regulation, among other things.

I don't think all of that is a bad thing (regulation tends to make it harder to do the first two things), but "free markets" are the economic equivalent to the "point mass" in physics: perhaps useful sometimes to create simple models and explanations of things, but will never exist in the real world.

iforgot2210• 1 year ago

The Nvidia export restrictions also might be shooting us in the foot too, or at least Nvidia. They really benefit from CUDA remaining the de facto standard.

onlyrealcuzzo9• 1 year ago

DeepSeek's technology is out of the bag.

Every LLM provider in the US will be using it to lower OpEx.

One of them is likely to pass those savings along to consumers to gain market share.

Facebook is in the business of providing weights for free.

The idea that we are all doomed unless we immediately migrate to DeepSeek is fantasy.

sangnoir• 1 year ago

Banned in the USA. Only.

The US government has the wherewithal to drag Europe along with it, like they did with Huawei's 5G equipment.

roblabla• 1 year ago

So far, a general tiktok ban (as opposed to a tiktok ban on things like government phones) has only been in effect in the USA. I highly doubt Europe would play ball at any attempt at banning imports of DeepSeek.

Besides, it's kinda too late for this. The model is freely accessible, so any attempt at banning it would be _completely_ moot. If DeepSeek keeps releasing their future models for free, I don't see how a ban could ever be effective at all. Worse case scenario, big tech can't use those models... but then individuals (and startups willing to go fast and break laws) will be able to use them and instantly get a leg up on the competition.

HotHotLava• 1 year ago

Used to. If they're going to start a trade war, pull out of NATO and invade Greenland instead, there'll not be much soft power left to drag Europe anywhere.

buyucu7• 1 year ago

And yet, Huawei is still doing fine.

realusername2• 1 year ago

Not really anymore, and I would bet that the way things are going, Twitter will be banned in the EU before Tiktok.

crote2• 1 year ago

Perhaps a few years ago, yes. But considering that the US president is now threatening to invade and annex part of a European country, I don't think past results can be extrapolated to the present.

buyucu7• 1 year ago

I have a Xiaomi phone, a Huawei Matebook and a BYD car. I guess I am pivoting to China after all :)

greenchair• 1 year ago

world is pretty small as we see with all nations kissing the ring in the last few weeks. pivot away from chinar will accelerate.

dang9• 1 year ago

We detached this subthread from https://news.ycombinator.com/item?id=42866072.

ks2048• 1 year ago

The schadenfreude and irony of this is totally understandable.

But, I wonder - do companies like OpenAI, Google, and Anthropic use each others models for training? If not, is it because they don't want to or need to, or because they are afraid of breaking the ToC?

Digit-Al• 1 year ago

So... company that steals other people's work to train their models is complaining because they think someone stole their work to train their models.

Cry me a river.

baggiponte• 1 year ago

OpenAI coping so hard

_moof• 1 year ago

This reminds me of a (probably apocryphal) story about fast food chains that made the rounds decades ago: McDonald's invests tons of time into finding the best real estate for new stores; Burger King just opens stores near McDonalds!

nazgulsenpai• 1 year ago

About 15 years ago, as CVS Pharmacy expanded into their new, stand-alone properties (in our region), Walgreen's Pharmacy started appearing across the street almost instantaneously. I've seen it happen at 4 separate locations so most certainly not coincidence -- so I believe it :)

dragonwriter3• 1 year ago

Hey, OpenAI, so, you know that legal theory that is the entire basis of your argument that any of your products are legal? "Training AI on proprietary data is a use that doesn't require permission from the owner of the data"?

You might want to consider how it applies to this situation.

buyucu7• 1 year ago

I have no sympathy for OpenAI here. They are (allegedly) a non-profit with open in the title that refuse to open-source their models.

They are now upset at a startup who is more loyal to OpenAI's original mission that OpenAI is today.

Please, give me a break.

Jotalea• 1 year ago

I really hate when there is a paywall to read an article. It makes me not want to read it anymore.

mbowcut2• 1 year ago

So, is this just an example of the first-mover disadvantage (or maybe the problem of producing public goods?). The first AI models were orders of magnitude more expensive to create, but now that they're here we can, with techniques like distillation, replicate them at a fraction of the cost. I am not really literate in the law but weren't patents invented to solve problems like this?

adam_arthur• 1 year ago

Who cares?

They did the exact same thing with public information. Their model just synthesizes and puts out the same information in a slightly different form.

Next we should sue students for repeating the words of their teachers

moralestapia4• 1 year ago

Called it from day 0, impossible to reach that performance with 5M, they had to distill OpenAI (or some other leading foundational model).

Got downvoted to oblivion by people who haven't been told what to think by MSM yet. Now it's on FT and everywhere, good, what matters is that truth comes out eventually.

I don't take any sides and think what DeepSeek did is fair play, however, what I do find harmful about this is, what incentive would company A have to spend billions training a new frontier model if all of that could be then reproduced by company B at a fraction of the cost?

kgeist3• 1 year ago

The "evidence" is very weak though:

The San Francisco-based ChatGPT maker told the Financial Times it had seen some evidence of “distillation”, which it suspects to be from DeepSeek.

Given that many people have been using ChatGPT to distill their fine-tunes for a few years now, how can they be sure it was specifically DeepSeek? There's, say, glaive.ai whose entire business model is to sell you synthetic datasets, probably generated with ChatGPT as well.

moralestapia4• 1 year ago

I agree that the evidence is weak, and even if they had some, they cannot really do anything.

To me, it's just very likely they distilled GPT-4, because:

1) Again, you just cannot get that performance at that cost. And no, what they describe on the paper is not enough to explain the 1,000x-fold decrease in cost.

2) Very often, DeepSeek tells you it's ChatGPT or OpenAI; it's actually quite easy to get it to do that. Some say that's related to "the background radiation on the post-AI internet". I'm not a fentanyl consumer so, unfortunately, I think that argument is trash.

suraci2• 1 year ago

Personally, I found deepseek is very very good at Chinese. I mean, it's highly literary and eloquent, it's quite amazing.

I didn't see this in o1 or any other LLM. Can distillation give deepseek such capability?

nachox999• 1 year ago

Ask DeepSeek and ChatGPT: "name three persons"; the answer may surprise you

curvaturearth• 1 year ago

Something about the outputs becoming the inputs to then produce more outputs is just plain funny

B1FF_PSUVM• 1 year ago

"Cry me a river" is a phrase I haven't heard recently, for some reason ...

asdefghyk• 1 year ago

Deepseek did not respect OpenAI's copyright?

Well who would have thought that?

nataliste• 1 year ago

A Wolf had stolen a Lamb and was carrying it off to his lair to eat it. But his plans were very much changed when he met a Lion, who, without making any excuses, took the Lamb away from him.

The Wolf made off to a safe distance, and then said in a much injured tone:

"You have no right to take my property like that!"

The Lion looked back, but as the Wolf was too far away to be taught a lesson without too much inconvenience, he said:

"Your property? Did you buy it, or did the Shepherd make you a gift of it? Pray tell me, how did you get it?"

What is evil won is evil lost.

m3kw9• 1 year ago

So if OpenAI didn't have these outputs for distillation, Deepseek wouldn't exist?

hedayet• 1 year ago

Beyond the irony of their stance, this reflects a failure of OpenAI's technical leadership—either in oversight or in designing a system that enables such behavior.

But in capitalism, we, the customers aren't going to focus on how models are trained or products are made; we only care about favourable pricing.

A key takeaway for me from this news is the clause in OpenAI's terms and conditions. I mistakenly believed that paying for OpenAI’s API granted full rights to the output, but it turns out we’re only buying specific rights (which is now another reason we're going to start exploring alternatives to OpenAI)

mtlmtlmtlmtl• 1 year ago

So, what is this evidence? I'll believe it when I see it. Right now all we really have is some vague rumours about some API requests. How many requests? How many tokens? Over how long of a time period? Was it one account or multiple, if the latter, how many? How do they know the activity came from deepseek? How do they know the data was actually used to train Deepseek models(could have just been benchmarking against the competition)?

If all they really have is some API requests, even assuming they're real and originated by Deepseek, that's very far from proof that any of it was used as training data. And honestly, short of commiting crimes against Deepseek(hacking), I'm not sure how they even could prove that at this point, from their side alone.

And what's even more certain is that a vague insistence that evidence exists, accompanied by a denial to shed any more light on the specifics, is about as informative as saying nothing at all. It's not like OpenAI and Microsoft have a habit of transparency and honesty in their communication with the public, as proven by an endless laundry list of dishonest and subversive behaviour.

In conclusion, I don't see why I should give this any more credence than I would a random anon on 4chan claiming a pizza place in Washington DC is the centre of a child sex trafficking ring.

P.S: And to be clear, I really don't care if it is true. If anything, I hope it is; it would be karmic justice at its finest.

halyconWays• 1 year ago

Oh no, so sad. The Open non-profit that steals 100% of all copyrighted content and makes multiple billion-dollar for-profit deals while releasing no weights is crying. This is going to ruin my sleep. :(

htrp3• 1 year ago

In other news.....water is wet

1propionyl• 1 year ago

At this point, the only thing that keeps me using ChatGPT is o1 w/ RAG. The usage limits on o1 are prohibitively tight for regular use, so I have to budget usage to tasks that would benefit there. I also have significant misgivings about their policies around output, which also limit what I can use it for.

For local tasks, the deepseek-r1:14b and deepseek-r1:32b distillations immediately replace most of that usage (prior local models were okay, but not consistently good enough). Once there's a "just works" setup for RAG on par with installing ollama (which I doubt is far of), I don't see much reason to continue paying for my subscription.

Sadly, like many others in this thread, I expect under the current administration to see self-hamstringing protectionism further degrade the US's likelihood of remaining a global powerhouse in this space. Betting the farm on the biggest first-mover who can't even keep up with competition, has weak to non-existent network effects (I can choose a different model or service with a dropdown, they're more or less fungible), has no technological moat and spent over a year pushing apocalyptic scenarios to drum up support for a regulatory moat...

...well it just doesn't seem like a great idea to me.

divbzero• 1 year ago

I was wondering if this might be the case, similar to how Bing’s initial training included Google’s search results [1]. I’d be curious to see more details of OpenAI’s evidence.

It is, of course, quite ironic for OpenAI to indiscriminately scrape the entire web and then complain about being scraped themselves.

[1] https://searchengineland.com/google-bing-is-cheating-copying...

schaefer• 1 year ago

I mean, if openAI claims they can train on the world’s novels and blogs with “no harm done” (i.e: no copyright infringement and no royalties due), then it directly follows that we can train both our robots and our selves on the output of openAI’s models in kind.

Right?

davesque• 1 year ago

I recently thought of a related question. Actually, I'm almost certain that foundation model trainers have thought of this. The question is to what extent are popular modern benchmarks (or any reference to them, or description of them, etc.) bring scrubbed from the training data? Or are popular benchmarks designed in such a way that they can be re-parametrized for each run? In any case, it seems like a surprisingly hard problem to deal with.

highfrequency2• 1 year ago

If true, the question is: did they use ChatGPT outputs to create Deepseek V3 only, or is the R1-zero training process a complete lie (given that the whole premise is that they used pure reinforcement learning)? If they only used ChatGPT output when training V3, then they succeeded in basically replicating the jump from ChatGPT-4o to o1 without any human-labeled CoT (and published the results) - which is a big achievement on its own.

henry_viii• 1 year ago

So Meta can train its AI on all the pirated books in the world but people are losing their mind over an AI learning from another AI?

esafak5• 1 year ago

People here have been vocal against training on any unlicensed content.

nuc1e0n• 1 year ago

And OpenAI scrapped the public internet to train its models.

boxedemp• 1 year ago

Deep refers to itself as ChatGPT sometimes lol

LZ_Khan• 1 year ago

I actually think what DeepSeek did will slow down AI progress. What's the incentive to spend billions developing frontier models if once it's released some shady orgs in unregulated countries can just scrape your model outputs, reproduce it, and undercut you in cost?

OpenAI is like a team of fodder monkeys stepping on landmines right now, with the rest of the world waiting behind them.

zx10rse• 1 year ago

OpenAI is already irrelevant but the audacity oh my.

cumulative00x• 1 year ago

There is a saying in Turkish that roughly goes like this, it takes a thief to catch a thief. I am not a big fan of China's tech, too, however, it amuses me to watch how big tech charlatans have been crying over Deepseek shock.

gosub100• 1 year ago

It's true irony to see thieves getting stolen from.

DidYaWipe• 1 year ago

They have "open" right in their name, so...

Objection overruled.

SubiculumCode• 1 year ago

If you have a set of weights A, can you derive another set of weights B that function (near) identically as A AND a) not appear to be the same weights as A when inspected superficially b) appear uncorrelated when inspecting the weight matrices?

rahimnathwani• 1 year ago

Do you mean for a given model structure, can two sets of weights give substantially the same outputs?

Even if that were possible, it would be suspicious if you were to release an open model whose model architecture is identical to that of a closed one from a competitor.

If that is what happened, we'd know about it by now.

fimdomeio• 1 year ago

But what is the problem here? Isn’t open AI mission “to ensure that artificial general intelligence benefits all of humanity”? Sounds like success to me.

hugoromano• 1 year ago

OpenAI initially scraped the web and later formed partnerships to train on licensed data. Now, they claim that DeepSeek was trained on their models. However, DeepSeek couldn't use these models for free and had to pay API fees to OpenAI. From a legal standpoint, this could be seen as a violation of the terms and conditions. While I may be mistaken, it's unclear how DeepSeek could have trained their models without compensating OpenAI. Basically, OpenAI is saying machines can't learn from their outputs as humans do.

buildsjets• 1 year ago

Womp Womp.

asdfasdf1• 1 year ago

it's no crime to steal from a thief

krapp• 1 year ago

It is actually a crime to steal from a thief.

wendyshu• 1 year ago

If distillation gives you a cheaper model with similar accuracy, why doesn't OpenAI distill its own models?

mkoubaa• 1 year ago

OpenAI made a lot of contributions to LLMs obviously but the amount of fraud, deception, and dark patterns coming out of that organization make me root against it.

kelseydh2• 1 year ago

The name itself, as for-profit closed source software, is grating.

ysofunny• 1 year ago

I see this as China fighting U.S. of A (or the American Dollar versus Chinese Renmibi if you will)

and this is good because any alternatives I can think of are older-school fighting

modern war is seeped in symbolism, but the contest is still there

e.g. whose dong is bigger? Xi Jingping's or Dnld Trump's

maxglute• 1 year ago

Not that DeepSeek is luigi mangione, but it's pretty funny OpenAi getting the dead ceo treatment.

mrkpdl• 1 year ago

The cat is out of the bag. This is the landscape now, r1 was made in a post-o1 world. Now other models can distill r1 and so on.

I don’t buy the argument that distilling from o1 undermines deep seek’s claims around expense at all. Just as open AI used the tools ‘available to them’ to train their models (eg everyone else’ data), r1 is using today’s tools.

Does open AI really have a moral or ethical high ground here?

ijidak3• 1 year ago

Plus, it suggests OpenAI never had much of a moat.

Even if they win the legal case, it means weights can be inferred and improved upon simply by using the output that is also your core value add (e.g. the very output you need to sell to the world).

Their moat is about as strong as KFC's eleven herbs and spices. Maybe less...

jamil72• 1 year ago

Agree 100%, this was also bound to happen eventually, OpenAI could have just remained more "open" from the beginning and embraced the inevitable commoditization of these models. What did delaying this buy them?

khazhoux6• 1 year ago

What did delaying this cost them, though? Hurt feelings of people here who thought OpenAI personally pledged openness to them?

plantwallshoe2• 1 year ago

I don’t think there is any ethical issue here, but I don’t think it’s good for the industry to remove all incentives for companies to spend lots of money solving hard, novel problems.

Why would anyone go through the effort of training the next groundbreaking model if they know they can just wait for someone else to do it and leverage that work?

csomar3• 1 year ago

Why would anyone go through the effort of training the next groundbreaking model if they know they can just wait for someone else to do it and leverage that work?

Why would anyone write, work or research anything if they know it would be consumed by AI and sold on a $xx/month subscription?

yapyap• 1 year ago

It sounds like they’re just jealous and trying to smear shit over the wall and see what sticks.

DeepSeek just bodied u bro, get back in the lab & create a better AI instead of all this news that isn’t gonna change them having a good AI

FpUser• 1 year ago

Pot calling kettle black?

almostdeadguy• 1 year ago

Hope Sam Altman is getting his money's worth out of that Trump campaign contribution. Glorious days to be living under the term of a new Boris Yeltsin. Pawning and strip-mining the federal apparatus to the most loyal friends and highest bidders.

ijidak3• 1 year ago

This whole argument by OpenAI suggests they never had much of a moat.

Even if they win the legal case, it means weights can be inferred and improved upon simply by using the output that is also your core value add (e.g. the very output you need to sell to the world).

Their moat is about as strong as KFC's eleven herbs and spices. Maybe less...

sirolimus• 1 year ago

Such Karma lol, I wonder how they trained Sora again? You..tube something

leobg2• 1 year ago

OpenAI is taking the position similar to that if you sell a cook book, people are not allowed to teach the recipes to their kids, or make better versions of them.

That is absurd.

Copyright law is designed to strike a balance between two issues. One the one hand, the creator’s personality that’s baked into the specific form of expression. And on the other hand, society’s interest in ideas being circulated, improved and combined for the common good.

OpenAI built on the shoulders of almost every person that wrote text on a website, authored a book, or shared a video online. Now others build on the shoulders of OpenAI. How should the former be legal but not the latter?

Can’t have it both ways, Sam.

(IAAL, for what it’s worth.)

otterley• 1 year ago

As another attorney, I would impart some more wisdom:

"Karma's a bitch, ain't it."

chris_wot• 1 year ago

I quite agree. The NY Times must be feeling a lot of schadenfreude right now.

hintymad3• 1 year ago

Just to play devil's advocate, OAI can argue that they spent great effort creating and procuring annotated data. Such datasets are indeed their secret, and now DS gets them for free by distilling OAI's output. Besides, OAI's EULA explicitly forbids users from using the output of their API for model training. I'm not saying that OAI is right, of course. Just to present OAI's point of view.

9dev• 1 year ago

So a bank robber that manages to steal from Fort Knox gets to keep the gold bars because it was a very complicated job?

mcmcmc• 1 year ago

They aren’t actually getting the dataset though.

kragen• 1 year ago

Feist comprehensively rejected that argument under US copyright law, and the attempts in the late 90s to pass a law in response establishing a sui generis prohibition on copying databases also failed in the US. The EU did adopt a directive to that effect, which may be why there are no significant European search engines.

However, OpenAI and Google are far more politically influential than the lobbyists in the 90s, so it is likely to succeed.

addicted7• 1 year ago

This is an incomplete version of OpenAI’s point of view.

OpenAI has a legally submitted point of view that they believe the benefits of AI to humanity are so great that anyone creating AI should be allowed to trample all over copyright laws, Terms of Use, EULAs, etc.

But OpenAI’s version of benefit to humanity is that they should be allowed to trample over those laws so they can benefit humanity by closely guarding the output of trampling those laws and charging humanity an access fee.

Even if we accept all of OpenAI’s criticisms of DeepSeek, they’re arguing that DeepSeek doing the exact same thing, but releasing the output for free for anyone to use is somehow less beneficial to humanity.

dkjaudyeqooe4• 1 year ago

You can't copyright AI generated works. OpenAI are barking up the wrong tree.

washadjeffmad• 1 year ago

They're not making a legal claim, they're trying to establish provenance over Deepseek in the public eye.

pclmulqdq• 1 year ago

They use their terms of service as both a sword and a shield here. It's a little bit ridiculous.

wzdd• 1 year ago

The stuff about copyright seems irrelevant.

OpenAI's future investments -- billions -- were just threatened to be undercut by several orders of magnitude by a competitor. It's in their best interests to cast doubt on that competitor's achievements. If they can do so by implying that OpenAI are in fact the source of most of the DeepSeek's performance then all the better.

It doesn't matter whether there's a compelling legal argument around copyright, or even if it's true that they actually copied. It just needs to be plausible enough that OpenAI can make a reasonable case for continuing investment at the levels it's historically attained.

And plausibility is something they've handily achieved with this announcement -- the sentiment on HN at least is that it is indeed plausible that DeepSeek trained on OpenAI. Which means there's now doubt that a DeepSeek-level model could be trained without making use of OpenAI's substantial levels of investment. Which is the only thing that OpenAI should be caring about.

tylerchilds2• 1 year ago

I’d take this argument more seriously if there weren’t billboards advocating hiring AI employees instead of human employees.

Sure, Open AI invested billions banking on the livelihood of every day people being replaced, or as Sam says, “A renegotiation of the social contract”

so as an engineer that is being targeted by meta and sales force under the “not hiring engineers plan” all o have to say to Open AI is “welcome to the social contract renegotiation table”

thayne2• 1 year ago

Even if that narrative is true, they were still undercut by DeepSeek. Maybe DeepSeek couldn't have succeeded without o1, but then it should have been even easier for OpenAI to do what DeepSeek did, since they have better access to o1.

chii• 1 year ago

It's in their best interests to cast doubt on that competitor's achievements.

it is, but the 2nd order logic says that if they are trying to cast doubt, it means they've got nothing better to offer and casting doubt is the only step they have.

if i was an investor in openAI, this should be very scary as it simply means I've overvalued it.

unclebucknasty7• 1 year ago

It just needs to be plausible enough that OpenAI can make a reasonable case for continuing investment at the levels it's historically attained
there's now doubt that a DeepSeek-level model could be trained without making use of OpenAI's substantial levels of investment.

But, this still seems to be a problem for OpenAI. Who wants to invest "substantially" in a company whose output can be used by competitors to build an equal or better offering for orders of magnitude less?

Seems they'd need to make that copyright stick. But, that's a very tall and ironic order, given how OpenAI obtained its data in the first place.

There's a scenario where this development is catastrophic for OpenAI's business model.

stingraycharles• 1 year ago

Isn’t this precisely how so many opensource LLMs caught up with OpenAI so quickly, because they could just train on actual ChatGPT output?

SecretDreams4• 1 year ago

OpenAI is going out of their way to demonstrate that they will willingly spend the money of their investors to the tune of 100s of billions of dollars, only to then enable 100s of derivative competitors that can be launched at a fraction of the cost.

Basically, in a round about way, OpenAi is going back to their roots and more - they're something between a charity and Robin Hood, stealing the money of rich investors and giving it to poor and aspirational AI competitors.

1vuio0pswjnm7• 1 year ago

"It doesn't matter whether there's a compelling legal argument around copyright, or even if it's true they actually copied."

Indeed, when the alleged infringer is outside US jurisdiction and not violating any local laws in the country where it's domiciled.

The fact that Microsoft cannot even get this app removed from "app stores" tells us all we need to know.

It will be OpenAI and others who will be copying DeepSeek.

Some of us would _love_ to see Microsoft try to assert copyright over a LLM. The question might not be decided in their favour, putting a spectre over all their investment. It is not a risk worth taking.

Anyone remember this one: https://en.wikipedia.org/wiki/Microsoft_Corp._v._Zamos

gerdesj• 1 year ago

I am not a lawyer (I'm UK based) You are a lawyer (probably local).

My understanding is that legal positions and arguments (within Common Law) need not be consistent across "cases" - they are considered in isolation with regards the body of law extant at the time.

I think that Sam can quite happily argue two differing points of view to two courts. Until a judgement is made, those arguments are simply arguments and not "binding" or even "influential elsewhere" or whatever the correct terms are.

I think he can legitimately argue both ways but may not have it both ways.

aleph_minus_one• 1 year ago

I think he can legitimately argue both ways

It would be very sensible that if a trial comes up, all these arguments that Sam Altman made for the other side score against him and OpenAI.

cscurmudgeon6• 1 year ago

OpenAI is taking the position similar to that if you sell a cook book, people are not allowed to copy the recipes into their own book and claim they did it all on their own.

leobg2• 1 year ago

Nobody is copying their model parameters or inference code.

What people “suck out” of their API are the general ideas. And they do it specifically so they can reassemble them in their own way.

It’s like reading all the Jack Reacher novels and then creating your own hero living through similar situations, but with a different name.

You’ll read it and you’ll say, dang, that situation/metaphor/expression/character reminds me of that Reacher novel. But there’s nothing Lee Child can do about it.

And that’s perfectly fine. Because he himself took many of his ideas from others, like Le Carré.

It’s the Eternal Tao.

Spooky233• 1 year ago

Seriously. OpenAI consciously stole The NY Times for almost all news content. Everything about the company is shady.

Sam should focus on the product instead of trying to out-jerk Elon and his buddies.

randombits0• 1 year ago

Recipes, that is, lists of ingredients and preparation instructions, are specifically uncopyrightable. Perhaps that’s why they used it as an example.

aucisson_masque• 1 year ago

I don't see the difference between that and LLM feeding on internet people's data.

They call it IP theft yet when the New York Times sued OpenAI and Microsoft for copyright infringement they claimed it's fair use of data.

duchenne• 1 year ago

The reasoning happens in the chain of thoughts. But OpenAI (aka ClosedAI) doesn't show this part when you use the o1 model, whether through the API or chat. They hide it to prevent distillation. Deepseek, though, has come up with something new.

manamorphic• 1 year ago

Crazy how most people miss this simple logical deduction.

mkayle• 1 year ago

This raises the same questions I have about OpenAI: where's all this data coming from, and do they have permission to use it?

beezlewax• 1 year ago

This is nothing short of hilarious.

whoknowsidont• 1 year ago

They can claim this all they want. But DeepSeek released the paper (several actually) on what they did, and it's already been replicated in other models.

It simply doesn't matter. Their methodology works.

EGreg• 1 year ago

Okay and there is evidence OpenAI used data of many people to train its own model.

Tell me again how come remixing our data is just dandy, many artists got disrupted — but no one should be able to disrupt OpenAI like that?

dbg31415• 1 year ago

Boo hoo?

Back in college, a kid in my dorm had a huge MP3 collection. And he shared it out over the network, and people were all like, "Man, Patrick has an amazing MP3 collection!" And he spent hours and hours ripping CDs from everyone so all the music was available on our network.

Then I remember another kid coming in, with a bigger hard drive, and he just copied all of Patrick's MP3 collection and added a few more to it. Then ran the whole thing through iTunes to clean up names and add album covers. It was so cool!

And I remember Patrick complained, "He stole my MP3 collection!"

Anyway this story sums up how I feel about Sam Altman here. He's not Metalica, he's Patrick.

https://www.npr.org/2023/12/27/1221821750/new-york-times-sue...

kranke155• 1 year ago

The very idea that OAI scrapes the entire internet and ignore individual rights and thats ok, but if another company takes the output data from their model, thats a gross violation of the law / TOS - that very idea is evil.

nbgoodall• 1 year ago

I lol'd, from the DeepSeek news release[1]: "Pushing the boundaries of open AI!"

[1] https://api-docs.deepseek.com/news/news250120

sgammon• 1 year ago

The nyt disclosure on this reporting is about to be wild

imchillyb• 1 year ago

If OpenAI desires public protection, then OpenAI should open-source its models.

If they did this, We the People would cover them like we do others. Without it, We the People don't care.

Cry, don't cry, it's meaningless to us.

xyst• 1 year ago

What a load of shit. ClosedAI is publishing a hit piece on DeepSeek and get public and politicians on their side. Maybe even get government to do their dirty work.

If they had a case, they wouldn’t be using FT. They would be filing a court case. Although that would open them up to discovery and the nasty shit ClosedAI has been up to would be game.

ddingus2• 1 year ago

So what?

Seriously. Given how pretty much all this software was trained, who cares?

I, for one, don't and believe the massive amount of knowledge continues to be of value to many users.

And I find the thought of these models knowing some things they shouldn't very intriguing.

esskay2• 1 year ago

Hard to really have any sympathy for OpenAI's position when they're actively stealing content, ignoring requests to stop then spending huge amounts to get around sites running ai poisoning scripts, making it clear they'll still take your content regardless of if you consent to it.

michaelmarkell• 1 year ago

Can someone with more expertise help me understand what I'm looking at here? https://crt.sh/?id=10106356492

It looks like Deepseek had a subdomain called "openai-us1.deepseek.com". What is a legitimate use-case for hosting an openai proxy(?) on your subdomain like this?

Not implying anything's off here, but it's interesting to me that this OpenAI entity is one of the few subdomains they have on their site

gkbrk• 1 year ago

Could just be an OpenAI-compatible endpoint too. A lot of LLM tools use OpenAI compatible APIs, just like a lot of Object Storage tools use S3 compatible APIs.

jongjong• 1 year ago

If the material which OpenAI is trained on is itself not subject to copyright protections, then other LLMs trained on OpenAI should also not be subject to any copyright restrictions.

You can't have both ways... If OpenAI wants to claim that the AI is not repeating content but 'synthesizing it' in the same was as a human student would do... Then I think the same logic should extend to DeepSeek.

Now if OpenAI wants to claim that its own output is in fact copyright-protected, then it seems like it should owe royalty payments to everyone whose content was sourced upstream to build its own training set. Also, synthetic content which is derived from real content should also be factored in.

TBH, this could make a strong case for taxing AI. Like some kind of fee for human knowledge and distributed as UBI. The training data played a key part in this AI innovation.

As an open source coder, I know that my copyrighted code is being used by AI to help other people produce derived code and, by adapting it in this way, it's making my own code less relevant to some extent... In effect, it could be said that my code has been mixed in with the code of other open source developers and weaponized against us.

It feels like it could go either way TBH but there needs to be consistency.

ripped_britches3• 1 year ago

I wish there were a stock ticker for OpenAI just to see what wall street’s take on all this is. One can imagine based on Nvidia, but I imagine OpenAI private valuation is hit much harder. Still, I think they’ll be able to justify it by building amazing products. Just interesting to watch what bankers think.

ripped_britches3• 1 year ago

There were definitely still very impressive engineering breakthroughs.

Also it’s pretty good confirmation that synthetic data is a valid answer to the data wall problem (non-problem).

therealpygon2• 1 year ago

“OpenAI complains company paid them for AI output that has no copyright, which was subsequently used to train another AI.” I think I fixed the title.

alasr• 1 year ago

OpenAI says it has evidence DeepSeek used its model to train competitor.
The San Francisco-based ChatGPT maker told the Financial Times it had seen some evidence of “distillation”, which it suspects to be from DeepSeek.
...
OpenAI declined to comment further or provide details of its evidence. Its terms of service state users cannot “copy” any of its services or “use output to develop models that compete with OpenAI”.

OAI share the evidence with the public; or, accept the possibility that your case is not as strong as you're claiming here.

janalsncm5• 1 year ago

Also, there are so many innovations in their papers (Deepseek math, Deepseek v2/v3, R1) that I honestly wouldn’t even care. They figured out a way to train on only 2048 H800s when big companies are buying them in the hundreds of thousands. They created a new RL algorithm. They improved MoE. They improved the KV cache. They built an super efficient training framework.

mercurialsolo• 1 year ago

How the vibe has turned on OpenAI?

coldpepper• 1 year ago

Fuck openai. They didn't ask my peemission to crawl my blog into their dataset.

karim792• 1 year ago

Oh God. I know exactly how this feels. A few years ago I made a bread hydration and conversion calculator for a friend, and put it up on JSFiddle. My friend, at the time, was an apprentice baker.

Just weeks later, I discovered that others were pulling off similar calculations! They were making great bread with ease and not having to resort to notebooks and calculators! The horror! I can't believe that said close friend of mine would actually share those highly hydraty mathematical formulas with other humans without first requesting my consent </sarc>.

Could it be, that this stuff just ends up in the dumpster of "sorry you can't patent math" or the like?

pshirshov• 1 year ago

A thief got robbed?..

a2128• 1 year ago

Yeah? And if I say I have evidence OpenAI used my data to train a competitor to myself as a being that's capable of programming, will I get to have my own story on the Financial Times?

jofzar• 1 year ago

Sorry, it's now a problem to train off other people's data? Surely openai has never trained off other people's data without permission...

TylerJaacks• 1 year ago

Cry me a fucking river OpenAI, as if your business model isn't entirely based on this exact same thing.

olalonde• 1 year ago

If it's true, how is it problematic? It seems aligned with their mission:

We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.
We will actively cooperate with other research and policy institutions; we seek to create a global community working together to address AGI’s global challenges.

https://openai.com/charter/

/s, we all know what their true mission is...

vcryan• 1 year ago

I love watching billionaires squirm

karim792• 1 year ago

So, banning high-powered chips to China has basically had the effect of turning them into extremophiles. I mean, that seems like a good plan </sarc>. Moreover, it is certainly slowing sales of one of the darling companies of the US (NVidia).

I just can't even begin to imagine what will come of this riduculous techno-imperialism/AI arms-race, or whatever you want to call it. It should not be too hard for China to create their own ASICs which do the same, and finally be done with this palaver.

kamranjon• 1 year ago

I was just wondering if this is even feasible?

The amount of iterations of training that would be needed for DeepSeek to actually learn anything from OpenAI would seem to be an insane amount of requests from a non-local AI, which you’d think would be immediately obvious to OpenAI just by looking at suspicious requests?

Am I correct in this assumption or am I missing something? Is it even realistic that something like this is possible without a local model?

emsign2• 1 year ago

"yOu ShOuLdN't TaKe OtHeR pEoPlE's DaTa!1!1" are they mental? How can people at OpenAI lack be so self-righteous and unaware? Is thia arrogance or a mental illness?

caseyy2• 1 year ago

Seeing as OpenAI is on the back foot, I hope nationalistic politicians don’t use this opportunity to strengthen patent laws.

If one could effectively patent software inventions, this would kill many industries, from video games (that all have mechanics of other games in them) to computing in general (fast algorithms, etc). Let’s hope no one gets ideas like that…

Granted, it would be ineffective in competing against China’s tech industry. But less effective laws have been lobbied through in the past.

exabrial• 1 year ago

cry us copyright holders a river.

rkagerer• 1 year ago

Are they crying about their competitor training off their stuff, after having used the whole of the web to train their own stuff?

bicepjai• 1 year ago

Reading this post, I can’t help but wonder if people realize the irony in what they’re saying. 1. “The issue is when you [take it out of the platform and] are doing it to create your own model for your own purposes,” 2. “There’s a technique in AI called distillation . . . when one model learns from another model [and] kind of sucks the knowledge out of the parent model,”

palisade• 1 year ago

Is this really the point OpenAI wants to start debating? When OpenAI steals everyone's data, it is fine. Right? But, let us pull the ladder up after that.

anon115• 1 year ago

eat shit

vjerancrnjak• 1 year ago

I thought this is capitalism for the winners. Why slander competition, just outcompete them? Why stick to your losing bets if you’ve recognized a better alternative?

Let’s race to the bottom.

seanp2k2• 1 year ago

"lol" said the Scorpion, "lmao".

oatmeal_croc• 1 year ago

Even if true, so what? These are increasingly looking like a competition between nation-states with their trade embargoes and export controls. All's fair in AI wars.

pknerd2• 1 year ago

OpenAI steals the data from Youtube and the Internet so that's no fair either.

MagicMoonlight• 1 year ago

So much for that walled garden. If rival firms can just download your entire model by talking to it then your company shouldn’t be worth billions.

ingohelpinger• 1 year ago

OpenAI should be quite, since they’ve scrapped the entire internet for their training data.

pknerd2• 1 year ago

The reason OpenAI is whining:

OpenAI’s o1 costs $60 per million output tokens; DeepSeek R1 costs $2.19. This nearly 30x difference brought the trend of falling prices to the attention of many people.

From Andrew Ng's recent DeeplearningAI newsletter

WolfOliver• 1 year ago

I guess DeepSeek payed OpenAI for the usage of their API according to OpenAI's pricing?

So what is the point if you pay for it and can not use the results how you see fit?

glooglork2• 1 year ago

How much data from o1 would DeepSeek actually need to actually make any improvements with it? I also assume they'd have to ask a very specific pattern of questions, is this even possible without OpenAI figuring out what's going on

fedeb95• 1 year ago

if some kind of transitivity holds, DeepSeek stole billions of internet users data.

krystofee• 1 year ago

I dont know if point of this is just to derail public attention to narative “hey, chinese stole our model, thats not fair, we need computee”, when the deepseek has clearly done some exceptional technical breakthrough on R1 and v3 models. Which even if you stole data from OpenAi is its thing.

thih9• 1 year ago

I don't mind and I believe that a company with "open" in its name shouldn't mind either.

I hope this is actually true and OpenAI loses its close to monopoly status. Having a for profit entity safeguarding a popular resource like this sounds miserable for everyone else.

At the moment AI looks like typical VC scheme: build something off someone else's work, sell it at cost at first, shove it down everyone's throats and when it's too late, hike the prices. I don't like that.

oysmal• 1 year ago

Given that the training approach was open sourced, their claim can be independently verified. Huggingface is currently doing that with Open R1, so hopefully we will get a concrete answer to whether these accusations are merited or not.

hello_computer• 1 year ago

then show it to us rachel

otikik• 1 year ago

Chatgpt, please generate an image of the tiniest violin imaginable.

Oh wait I will ask DeepSeek instead.

xinayder3• 1 year ago

OpenAI declined to comment further or provide details of its evidence. Its terms of service state users cannot “copy” any of its services or “use output to develop models that compete with OpenAI”.

Well, this sounds like they are just crying because they are losing the race so far. Besides, DeepSeek explicitly states they did a study on distillation on ChatGPT, then OpenAI is like "oh see guys they used our models!!!!!"

khazhoux6• 1 year ago

By what metric are they losing?

xinayder3• 1 year ago

DeepSeek is a fraction of the cost of ChatGPT, they needed far few resources than OpenAI. This is essentially what caused the massive selloff in Nvidia, as a new competitor model is just as good and requires a fraction of the massive costs.

I don't remember the correct metric but the cost for DeepSeek was like $15/mo while ChatGPT was $200

khazhoux6• 1 year ago

I'm disappointed that 99% of the comments about this topic are Schadenfreude, and 1% is actually about the technical implications of OpenAI's claims.

iimaginary• 1 year ago

Where did I leave my tiny violin?

janalsncm5• 1 year ago

I think readers should note that the article did not provide any evidence for OpenAI’s claims, only OpenAI declining to provide evidence, various people repeating the claim, others reacting to it.

It does matter whether it happened and how much it happened. Deepseek ran head to head comparisons against O1 so it would be pretty reasonable for them to have made API calls, for example.

But also, as the article notes, distillation, supervised fine tuning, and using LLM as a judge are all common techniques in research, which OpenAI knows very well.

trkaky• 1 year ago

how much would it cost to distill o1..

paul_e_warner4• 1 year ago

There seem to be two kinda incompatible things in this article: 1. R1 is a distillation o1. This is against it's terms of service and possibly some form of IP theft. 2. R1 was leveraging GPT-4 to make it's output seem more human. This is very common and most universities and startups do it and it's impossible to prevent.

When you take both of these points and put them back to back, a natural answer seems to suggest itself which I'm not sure the authors intended to imply: R1 attempted to use o1 to make its answers seem more human, and as a result it accidentally picked up most of it's reasoning capabilities in the process. Is my reading totally off?

ryao• 1 year ago

Given that OpenAI model outputs are littering the internet, is it even possible to train a new model on public webpages without indirectly using OpenAI’s model to train it?

udev4096• 1 year ago

What about the pirated books you used and millions of blogs and websites scraped without consent? Somehow that's legal? Come on, give me a fucking break. OpenAI deserves the top spot in the list of unethical companies in the world

oli5679• 1 year ago

this is pretty ridiculous

A. below is a list of OpenAI initial hires from Google. It's implausible to me that there wasn't quite significant transfer of Google IP

B. google published extensively, including the famous 'attention is all you need' paper, but open-ai despite its name, has not explained the breakthroughs that enabled O1. It has also switched from a charity to a for-profit company.

C. Now this company, with a group of smart, unknown machine learning engineers, presumably paid fractions of what OpenAI are published, has created a model far cheaper, and openly published the weights, many methodological insights, which will be used by OpenAI.

1. Ilya Sutskever – One of OpenAI’s co-founders and its former Chief Scientist. He previously worked at Google Brain, where he contributed to the development of deep learning models, including TensorFlow. 2. Jakub Pachocki – Formerly OpenAI’s Director of Research, he played a major role in the development of GPT-4. He had a background in AI research that overlapped with Google’s fields of interest. 3. John Schulman – Co-founder of OpenAI, he worked on reinforcement learning and helped develop Proximal Policy Optimization (PPO), a method used in training AI models. While not a direct Google hire, his work aligned with DeepMind’s research areas. 4. Jeffrey Wu – One of the key researchers involved in fine-tuning OpenAI’s models. He worked on reinforcement learning techniques similar to those developed at DeepMind. 5. Girish Sastry – Previously involved in OpenAI’s safety and alignment work, he had research experience that overlapped with Google’s AI safety initiatives.

throwaway3141553• 1 year ago

A. below is a list of OpenAI initial hires from Google. It's implausible to me that there wasn't quite significant transfer of Google IP

I agree there's hypocrisy but in terms of making a strong argument, you can safely remove your list of persons who (drum roll)... mostly _didn't_ actually work at Google?

dumah• 1 year ago

my_ridiculous_list = ["Ilya Sutskever"]

kozikow• 1 year ago

Chatgpt content is getting pasted all over the web. Now, for anyone crawling the web, it's hard to not include some chatgpt outputs.

So even if you put some "watermarks" in your AI generation, it's plausible defense to find publicly posted content with those watermarks.

Maybe it's explained in the article, but I can't access it, as it's paywalled.

cratermoon5• 1 year ago

Maybe the VCs backing OpenAI invest in tiny violins.

gejose• 1 year ago

Reminds me of this quote by Bill Gates to Steve Jobs, when Jobs accused Gates of stealing the idea for a mouse:

"Well, Steve… I think it’s more like we both had this rich neighbour named Xerox and I broke into his house to steal the TV set and found out that you had already stolen it."

witnesser2• 1 year ago

Soon another layer of distiller will emerge. Selling purer booze in this weight tuning buzzi.

rochak• 1 year ago

Cry me a river

low_tech_love• 1 year ago

Imagine having no competition one day and the next DeepSeek happens. It must’ve been quite scary. Makes sense that accusations will start flying. In my country we have a saying: a thief that robs a thief is pardoned for 100 years.

It’s really interesting that the same people who defend liberal capitalism at its extreme and praise competition as its most important component (which I don’t disagree) are the same ones that’ll promptly attempt to destroy the system and the competition as soon as they are in such a position.

sabhiram• 1 year ago

The grapes are sour because their moat is crumbling.

What was supposed to be a model, training, and data moat - is now reduced to operational cost, which they are not terribly efficient for.

OpenAI has been on a journey to burn as much $ as possible to get as far ahead on those three moats, to the point where decreasing TCO for them on inference was not even relevant - "who cares if you save me 20% of costs when I can raise on a 150b pre money value?".

Well, with their moats disappearing, they will have no choice but to compete on inference cost like everyone else.

juliuskiesian• 1 year ago

The obvious question is, if you have the evidence, why not just show it?

elzbardico3• 1 year ago

I used OpenAI APIs to generate training data for some run-of-the-mill ML models at my work, for some use cases where people wanted to use LLMs directly, but that could be easily fulfilled by smaller well trained models.

Is OpenAI going to complain about me too?

elzbardico3• 1 year ago

China is a society mostly run by engineers, some 70% of the CCP Politburo are STEM people by their formation. Engineering is a high prestige profession.

The West is run by lawyers, MBAs and salesmen.

This kerfuffle is a delicious study about this.

zhenghao1• 1 year ago

All I see is sour grapes. Can't stand someone else coming up with a far more superior and cheaper alternative. This is business dude. There's always going to be some new disruptor to shake the market up.

alexfromapex• 1 year ago

The public probably thinks that these companies are getting hacked by "sophisticated hackers" but I'd bet money that they've been hacked via social engineering.

freejazz• 1 year ago

Who else cares?

animitronix• 1 year ago

Who tf cares?

kapad• 1 year ago

Aah. So OpenAI can use whatever means necessary to gather data for training it's model. Regardless of copyright.

But somehow, it's a problem if another model developer distills it's model by training it on OpenAI?

IMO, if the first use is fair, then so is the second use.

OpenAI says it has evidence DeepSeek used its model to train competitor

Related Stories