In the LLM space, "open source" is being used to mean "downloadable weights"

https://www.alessiofanelli.com/blog/llama2-isnt-open-source

by FanaHOVA • 3 years ago

400 228 3 years ago

spullara2• 3 years ago

It remains to be seen in court whether weights are even copyrightable potentially making all the various licenses and their restrictions moot.

dvdkon3• 3 years ago

In a similar vein, the common "you may not use this model's output to improve another model" clause is AFAIK unenforceable under copyright, so it's at best a contractual clause binding a particular user. Anyone using that improved model afterward is in the clear.

taneq5• 3 years ago

And of course anyone using a model improved by this is entirely unworried by these clauses if their improved model takes off hard.

banana_feather2• 3 years ago

The idea is that if you violate the terms of the license to develop your own model, you lose your rights under the license and are creating an infringing derivative work. If I clone a GPL'd work and ship a derivative work under a commercial license, downstream users can't just integrate the derivative work into a product without abiding by the GPL terms and say "well we're downstream relative to the party who actually copied the GPL'd work, so the GPL terms don't apply to us".

ljdcfsafsa2• 3 years ago

it's at best a contractual clause binding a particular user. Anyone using that improved model afterward is in the clear.

That's... not really accurate. See the concept of tortious interference with a contract.

weinzierl• 3 years ago

I find the idea that weights are not copyrightable very fascinating - appealing even. I have a hard time imagining a world where this is the case, though.

Can you summarize why weights would not be copyrightable or give me pointers to sources that support that view.

slimsag2• 3 years ago

Speculating (I am not a lawyer) I see two options:

1. Model weights are the output of mathematical principles, in the US facts are not copyrightable, so in general math is not copyrightable.

2. Model weights are the derivative work of all copyrighted works it was trained on - in which case, it would be similar to creating a new picture which contains every other picture in the world inside of it. Who is the copyright owner? Well, everyone, since it includes so many other copyright holders' works in it.

sebzim45003• 3 years ago

Generally the output of a machine is not copyrightable. Similarly, the contents of a phone book is not copyrightable in the US even if the formatting/layout is. So I could take a phonebook and publish another one with identical phone numbers as long as I laid it out slightly differently.

paxys2• 3 years ago

It's a complicated question and I don't think anyone can give a clear yes or no answer before some court has ruled on it. One school of thought is that copyright is designed to protect original works of creativity, but weights are generated by an algorithm and not direct human expression. AI generated art, for example, has already been ruled ineligible for copyright.

appplication• 3 years ago

Let’s take a simple linear regression model with a handful of parameters. The weights could be an array of maybe 5 numbers. Should that be copyrightable? What if someone else uses the same data sources (e.g. OSS data sets) and architecture and arrives at the same weights? Is this a Copyright violation?

Let’s talk about more complex models. What if my model shares 5% of the same weights with your model? What about 50%? What about 99%? How much do these have to change before you’re in the clear? What if I take your exact model and run it through some extra layers that don’t do anything, but dilute the significance of your weights?

It’s a murky area, and I’m inclined to think copyright is not at all the right tool to handle the legality of these models (especially given the glaring irony they are almost all trained using copyrighted material). Patents, perhaps better suited, but I’m also not sold.

cbm-vic-202• 3 years ago

An analog to this might be the settings of knobs and switches for an audio synthesizer, or guitar effects settings. If you wanted to get the "Led Zeppelin sound" from a guitar, you could take a picture of the knobs on the various pedals and their configuration, and replicate that yourself. You then create a new song that uses those settings. Is that something that is allowed under copyright?

What if there were billions of knobs, tuned after years of feedback and observations of the sound output?

rvcdbn7• 3 years ago

I have a hard time imagining a world where it is not the case at least in the US i.e. where copyright is extended to a work with no originality in direct contradiction to copyright clause in the constitution.

humanistbot4• 3 years ago

And it also remains to be seen if various legislatures will pass laws that explicitly declare the copyright status of model weights. It is important to remember that what is or is not copyrightable can change.

rvcdbn7• 3 years ago

At least in the US copyright is established by the constitution so not sure how much it’s possible to change via the normal legislative process.

ljdcfsafsa2• 3 years ago

1. Why wouldn't they be and 2. Does that even matter? If you enter into a contract saying don't do X, and you do X, you're violating the contract.

sebzim45003• 3 years ago

I assume GP was talking about a scenario in which you had not entered into a contract with Meta. E.g. if I just downloaded the weights from someone else.

rvcdbn7• 3 years ago

1 - because they lack originality, see: https://constitution.congress.gov/browse/essay/artI-S8-C8-3-...

FanaHOVA5• 3 years ago

Yep, same with SSPL. GPL has been tested in FSF vs Cisco (2008), but none of the more restrictive licenses have.

jrockway3• 3 years ago

It seems like a dangerous clause to me.

1) "Dear artists, the model cannot infringe upon your copyright because it's merely learning like a human does. If it accidentally outputs parts of your book, you know, it just accidentally plagiarized. We all do it haha! Our attorneys remind you that plagiarism is not illegal in the US."

2) "Dear engineers, the output of our model is copyrighted and thus if you use it to train your own model, we own it."

I am not sure how both of those can be true at the same time.

jimmaswell• 3 years ago

We all truly do "accidentally plagiarize", especially artists. Many guitarists realize they accidentally copied a riff they thought they'd come up with on their own for example.

luma• 3 years ago

2) doesn't line up with the US court's current stance that only a human can hold copyright, and thus anything created by a not-human cannot have copyright applied. This applies to animals, inanimate objects, and presumably, AI.

I have no idea how this impacts the encodability of the license from FB which may rely on things other than copyright, but as of right now, the output absolutely cannot be copyrighted.

sangnoir2• 3 years ago

If they are nor copyrightable, that'll be the end of publicly-released weights by for-profit companies. All subsequent models will be served behind an API.

dragonwriter5• 3 years ago

If they are nor copyrightable, that'll be the end of publicly-released weights by for-profit companies

I don’t see why, for-profit companies release permissively-licensed ooen-source code all the time, and noncopyrightable models aren't practically much different than that.

taneq5• 3 years ago

There's "open source" in the original sense, where the source was available. Then there's "FOSS" where the source is not only available, but it's under a copyleft license designed to protect the IP from greedy individual humans. And then there's "open" in the Shenzhen sense where you can find the source and other data online and nobody's going to stop you building something based on those. This is an interesting timeline.

hiatus• 3 years ago

And then there's "open" in the Shenzhen sense where you can find the source and other data online and nobody's going to stop you building something based on those.

I believe there is a name for that: gongkai. https://www.bunniestudios.com/blog/?page_id=3107

taneq5• 3 years ago

Ooh, thanks! I've watched a few of bunnie's things in the past but that's a term I'll remember.

pzo• 3 years ago

On top of that there are also different OSS such as Apache and MIT that the latter one can still restrict the user from using because project owner might patented some algorithm and MIT license doesn't have patent grant.

LGPL3.0 also pretty much is restricted in a way that not sure if can be used to distribute software in App Store for iOS legally.

risho• 3 years ago

The original sense of open source is defined by the people who fractured off from the Free Software movement in the mid 90's and created it. It's just "Free Software" that has a focus on practicality and utility rather than "Free Software"'s focus on idealism and doing the right thing. It has NOTHING to do with "source available" which is a movement that has recently been co-opting the open source name.

"FOSS" has absolutely no requirement of it being copyleft. The MIT license is just as FOSS as the GPL. Many of the free software advocates do have an affinity for copyleft, but they are not mutually exclusive. There are plenty of FOSS advocates who also use and advocate for permissive licenses as well.

jordigh• 3 years ago

There's "open source" in the original sense

That original sense never existed. Virtually nobody said "open source" before OSI's 1998 campaign for "Open Source", as bankrolled by Tim O'Reilly.

https://thebaffler.com/salvos/the-meme-hustler

I know it's been a long time, and we've forgotten, but there is virtually no record of anyone saying "open source" before 1998, except in rare and obscure contexts and often unrelated to the modern meaning.

teddyh• 3 years ago

There’s this one from September 10th, 1996, which I find intriguing:

https://web.archive.org/web/20180402143912/http://www.xent.c...

api2• 3 years ago

I'm not sure open source applies to actual models. Models aren't human readable, so it's closer to a binary blob. It would apply to the training code and possibly data set.

Llama2 is a binary blob pre-trained model that is useful and is licensed in a fairly permissive way, and that's fine.

politelemon• 3 years ago

Yes I think you've put it well. If models were smaller I'd see those in the Github releases section. The model training is what I'd see in the source code and the README etc, to arrive at the 'blob'.

api2• 3 years ago

Even if it costs millions in compute to run at that scale, seeing that code would be extremely informative.

cjdell• 3 years ago

Very like a binary blob. You have to execute it to use it and impossible for humans to reason about just by looking at it.

At least binary blobs can be disassembled.

pk-protect-ai2• 3 years ago

llama2 is absolutely useless. From the small models the guanaco-33b and guanaco-65b are the best (though they are derived from llama).

ngai_aku• 3 years ago

Is that just based on your experience, or do you have a link to benchmarks?

pk-protect-ai2• 3 years ago

Try these prompts with different models. LLaMA 2 output is pure garbage: ----1---- On a map sized (256,256), Karen is currently located at position (33,33). Her mission is to defeat the ogre positioned at (77,17). However, Karen only has a 1/2 chance of succeeding in her task. To increase her odds, she can: 1. Collect the nightshades at position (122,133), which will improve her chances by 25%. 2. Obtain a blessing from the elven priest in the elven village at (230,23) in exchange for a fox fur, further increasing her chances by additional 25% Foxes can be found in the forest located between positions (55,33) and (230,90).

Find the optimal route for Karen's quest which maximizes her chances of defeating the ogre to 100%. ----2---- Write a python code using imageio.v3 to create a PNG image representing the map way-points and the route of Karen in her quest, each way-point must be of a different color and her path must be a gradient of the colors between the waypoints. ------------

I have a lot of cases those I test against different models ... GPT-4 since one week is really degraded, GPT-3.5 became a little bit better, and LLaMA2 is garbage.

Oranguru• 3 years ago

Useless for what? Are you comparing the base model with chat-tuned models?

Chat-tuned derivatives of LLaMa 2 are already appearing. Given that the base LLaMa 2 model is more efficient than LLaMa 1, it is reasonable to expect that these more refined chat-tuned versions of the chat-tuned versions will outperform the ones you mention.

monlockandkey• 3 years ago

wait for the tuned models

mk_stjames• 3 years ago

In the diagram, there is theoretically another category outside the 'Restricted Weights' but maybe less than the 'Completely Closed' superspace, and that would be something along the lines of 'Blackbox weights and model' that is free to use but essentially non inspectable or transferrable. This would be the sister to 'free to use' closed-source software. An AI that is free to use but provided as a binary blob would meet this criterion. Or a module importable to python that calls precompiled binaries for the inference engine + weights with no source available. The traditional complement of this in the current software world would be Linux drivers from 3rd parties that are not open source. They are free, but not open.

We haven't seen this too much yet in the AI world, as mostly people who open the weights are doing so in a research manner, where the inference is decidedly needed to be open sourced- and people with closed models do so in order to make money and thus no reason to open source the inference side either, just charge for an API ("OpenAI").

FanaHOVA5• 3 years ago

Yea I didn't include it, but that'd be the "free as in beer, but not freedom" circle :)

jerf• 3 years ago

This isn't really new, the strict "Open Source" as defined for software has never made exact, perfect sense for anything other than software. That's why the Creative Commons licenses exist; putting a photographic image under GPL2 has never made any sense. It always needs redefinition in new media.

Flimm• 3 years ago

It doesn't need redefinition. We just need a new term for new media.

not2b• 3 years ago

An LLM is more like software than it is like media. The GPL defines source code as the preferred form for making modifications, including the scripts needed for building the executable from source. The weights in this case are more similar to the optimized executable code that comes out of a flow. The "source" would be the training data and the code and procedures for turning that into a model. For very large LLMs almost no one could use this, but for smaller academic models it might make sense, so researchers could build on each others' work.

hardolaf• 3 years ago

The strict "Open Source" wasn't even a definition when I started college.

asadotzler2• 3 years ago

Open Source didn't exist until Netscape created Mozilla in early '98 and the definition was soon after developed and then for some years tuned until we have today's "Open Source".

RobotToaster• 3 years ago

Creative commons has never claimed to be an open source licence though, they usually use the term free culture.

alerighi2• 3 years ago

Even for medias such as photos, songs, videos, you have a source. That is the raw materials and the projects from which you rendered the image, the video or the audio output.

The source of a language model is more in reality the model, that is the code that was used to train the particular model. The model itself is more of a compiled binary, altough not in machine code.

So for a model to be really open source to me it would mean that you have to release the software used for generating it, so I can modify it, train it on my data, and use it.

skybrian• 3 years ago

I don't see why the term "open source" needs to evolve when "source available" is available. Or in this case, "weights available under a license with few restrictions."

mhh__• 3 years ago

New generation of programmers can't remember not having open source / free software of any kind so the difference is academic versus felt.

donretag• 3 years ago

Since Open Source has been established in the tech ethos for a while now, any deviation has been met with derision. It seems like the community has been more tolerant of these "open" licenses as of late. While must of the hate for projects that do not fit the FOSS standard is mostly unwarranted, hopefully we are not moving quickly in the "open" direction.

Here is another article on LLaMa2: https://opensourceconnections.com/blog/2023/07/19/is-llama-2...

rapatel0• 3 years ago

Fully reproducible model training might simply not be possible if information from the training environment is not captured. In addition to data and code you might have additional uncertainty from:

- pseudo/true random number generator and initialization

- certain speculative optimizations associated with training environments (distributed)

- Speculative optimizations associated with model compression

- Image decompression algorithm mismatch (basically this is library versioning)

- ....things I'm forgetting...

It's just a lot of things to remember to capture, communicate, and reproduce.

martincmartin• 3 years ago

pseudo/true random number generator and initialization

It's not just the generator and initialization. If you do anything multithreaded, like a producer/consumer queue, then you need to know which pieces of work went to which thread in which order.

It's a lot like reproducing subtle and rare race conditions.

monocasa• 3 years ago

Most of the mature ML environments are pretty focused on reproducible training though. It's pretty necessary for debugging and iteration.

curtis3389• 3 years ago

Part of the benefit of FOSS & open source is that a curious user can inspect how something is made and learn from it. It matters that open weights are no different from a compiled program. Sure, you can always modify an executable's instructions, but there's no openness there.

Then there's the problems of the content of the training data, which parallel the dangers of opaque algorithms.

Taek• 3 years ago

I didn't realize that the llama license forbids you from using its outputs to train other models. That's essentially a dealbreaker, synthetic data is going to be the most important type of training data from here on out. Any model that prohibits use of synthetic data to train new models is crippled.

SanderNL• 3 years ago

Good luck enforcing that, though. How would they ever know?

denlekke2• 3 years ago

i wonder if they could include some marker prompt and response that wouldn't occur "naturally" from any other model or training data

criddell• 3 years ago

Disgruntled current or former employee turning in their employer for the reward? That’s how Microsoft and the BSA used to bust people before the days of always online software.

danShumway10• 3 years ago

I don't see how this would be enforceable in law without killing almost every AI company on the market today.

The whole legal premise of these models is that training on copyrighted material is fair use. If it's not, then... I mean is Facebook trying to claim that including copyrighted material in a dataset isn't fair use regardless of the author's wishes? Because I have bad news for LLaMA then.

"You need permission to train on this" is an interesting legal stance for any AI company to take.

doctorpangloss• 3 years ago

The whole legal premise of these models is that training on copyrighted material is fair use.

Not to diminish the conversation here, but not even a Supreme Court Justice knows what the legality is. You’d have to be a whole 9 person Supreme Court to make an accurate statement here. I don’t think anyone really knows how Congress meant today’s laws to work in this scenario.

rcxdude3• 3 years ago

From my non-legal-professional POV I can see an angle which may work:

Firstly, llama is not just the weights, but also the code alongside it. The weights may or may not be copyrightable, but the code is (and possibly also the network structure itself? that would be important if true but I don't know if it would qualify).

Secondly, you can write what you want in a copyright license: you could write that the license becomes null and void if the licensee eats too much blue cheese if you want.

Following from that, if you were to train on the outputs of the AI, you may not be guilty of copyright infringement in terms of doing the training (both because AI output is not copyrightable in the first place, something which seems pretty set in precedent already, and possibly also because even if it was, it gets established that it is fair use like any other data), but if it means your license to the original code is revoked then you will at the very least need to find another implementation that can use the weights, or (if the weights can be copyrighted, which I would argue is probably not the case, if you follow the argument that the training is fair use, especially if the reasoning is that the weights are simply a collection of facts about the training data, but it's very plausible that courts will rule differently here).

This could wind up with some strange situations where someone generating output with the intent of using it for training could be prosecuted (or at least forced to cease and desist) but anyone actually using that output for training would be in the clear.

I agree it is extremely "have your cake and eat it" on the part of the AI companies: They wish to both bypass copyright and also benefit from the restrictions of it (or, in the case of OpenAI, build a moat by lobbying for restrictions on the creation and use of the models themselves, by playing to fears of AI danger).

Ajedi32• 3 years ago

I'd say it's enforceable in the sense that if you agree to the license then violating those terms would be breach of contract regardless of whether use of the LLaMA v2 output is protected by copyright or not. But there's nothing stopping someone else who didn't agree to the license from using output you generate with LLaMA v2 to train their model.

redox99• 3 years ago

It's so hypocritical, it's insane.

"Yes, we train our models on a good chunk of the internet without asking permission, but don't you dare train on our models' output without our permission!"

And OpenAI also has a similar restriction.

alerighi2• 3 years ago

In fact they can't (both Facebook and OpenAI) train their models without asking permission. Just wait for someone to start raising this concern. The EU is working on regulating these kind of aspects, for example this is not compliant at all with the GDPR (unless you train only on data that doesn't contain personal data, that is more rare than you would think).

6gvONxR4sf7o2• 3 years ago

It's hilarious that big players in this space seem to think these are consistent views:

- It's okay to train a model on arbitrary internet data without permission/license just because you can access it

- It's not okay train a model on our model

whatshisface• 3 years ago

The belief that makes them consistent is that the authors of a million Reddit posts have no way to assert their rights while the big company that trained a Redditor model does.

rahkiin• 3 years ago

Like google is allowed to scrape the whole internet but you’re not allowed to scrape google. Rules for thee but not for me

realusername• 3 years ago

Yes, they have to pick one or the other. Until then I'm going to assume that the model licence doesn't apply since the first point would be invalid and the model could not be built in the first place.

torstenvl6• 3 years ago

Those are perfectly consistent, despite what ideologically-driven people may want to believe.

Copyright is literally the right to copy. Arbitrary Internet data that is not copied does not have any copyright implications.

The difference is that LLaMa imposes additional contractual obligations that, for ideological reasons (Freedom #0), open source software does not.

This issue reminds me of the FSF/AGPL situation. At some point you just have to accept that copyright law, in and of itself, is not sufficient to control what people do with your software. If you want to do that, you have to limit end-user freedom with an EULA.

If someone uses LLaMa output to train models, it is unlikely they will be sued for copyright infringement. It is far more likely they will be sued for breach of contract.

lhnz• 3 years ago

It tells you that they think their moat is data quality/quantity.

taneq5• 3 years ago

I can license my LLM however I want to... but I can't sail this ship to generally-intelligent-Tortooga on me lonesome. Savvy?

chii• 3 years ago

think these are consistent views

they are consistent, if they believe themselves to be "special" and deserves special treatment!

moffkalast2• 3 years ago

I'm not sure why anyone would even do that in the first place, LLama doesn't generate synthetic data that would be even remotely good enough. Even GPT 3.5 and 4 are already very borderline for it, with lots of wrong and censored answers. And at best you make a model that's as good as LLama is, i.e. not very.

jstarfish• 3 years ago

Instruction-tuning is the obvious use case. That much has nothing to do with subjectivity, alignment or censorship, it's will-you-actually-show-this-as-JSON-if-asked.

zarzavat• 3 years ago

A contract ordinarily has to have consideration. Since LLaMa weights are not copyrightable by Meta and are freely available, what exactly is the consideration? The bandwidth they provide?

objektif• 3 years ago

I played with Llama2 for a bit and for a lot of the questions I asked I got complete made up garbage stuff. Why would you want to train on it?

lolinder• 3 years ago

Not that it's okay for this to be in the license, but I'm curious: what is the use case for synthetic data? Most of the discussion I've seen has been about how to avoid accidentally using LLM-generated data.

lmeyerov• 3 years ago

Tuning a tiny classifier

int_19h4• 3 years ago

You can use synthetic data produced by more complex models to finetune smaller ones to be better.

Der_Einzige4• 3 years ago

It's exactly the opposite. We have better ways to combine the knowledge of several models together than sampling them. (i.e. mixture of experts, model merges, etc) Relying on synthetic data from one LLM to train another LLM is in general a terrible idea and will lead to a race to the bottom.

tasubotadas• 3 years ago

Generate data using ai, save it, it cannot be copyrighted or anything, data isn't a model, use it as much as you want for training.

Ezpz

concinds• 3 years ago

Fundamentally untrue, and disheartening that it's the top comment.

You can't use a model's output to train another model, it leads to complete gibberish (termed "model collapse"). https://arxiv.org/abs/2305.17493v2

And the Llama 2 license allows users to train derivative models, which is what people really care about. https://github.com/facebookresearch/llama/blob/main/LICENSE

rgoldste• 3 years ago

The truth is between these two. You can use a model’s output to train another model, but it has drawbacks, including model collapse.

int_19h4• 3 years ago

Some of the best LLaMA finetunes today are trained on GPT-4 output.

Yes, you cannot do this kind of thing indefinitely and expect endless improvements from "endless training set". But that's a very different problem.

dheera• 3 years ago

forbids you from using its outputs to train other models.

I don't know how one can even forbid this. As a human, I'm a walking neural net, and I train myself on everything that I see, without a choice. The only difference is I'm a carbon-based neural net.

renewiltord• 3 years ago

I would just do it anyway. In fact, I can release a suitably laundered version and you'd never know. If I release a few million, each with slight variation, there's no way provenance can be established. And then we're home-free.

failuser2• 3 years ago

Of course, it’s not open source. With proliferation of the cloud, software has obtained an entirely new level of closeness: not being able to see the program binaries. Having an ability to run locally is now somewhat open in comparison.

Eduard• 3 years ago

An understood term like "open open" source shouldn't be hijacked and exploited for marketing purposes.

What these models do, they should either invented a new term, or use an appropriate existing term, eg. "fair use"

failuser2• 3 years ago

Absolutely. Maybe the term is already coined, but I don’t know it. Open source implies the ability to compile software from human-generated inputs. This is just self-hosted freeware.

flir• 3 years ago

"Nyet! Am not open source! Not want lose autonomy!"

(Downvotes... oops. The reference is Charlie Stross's Accelerando. The protagonist has a conversation with an AI that's just trying to survive. One of the options he suggests is to open source itself. Which is a roundabout way of saying that eventually we're going to have to take the AI's own opinions into account. What if it doesn't want to be open source?)

taneq5• 3 years ago

This post deserved better treatment, along with maybe a couple of metaphorical decerebrated kittens on its doorstep.

sytse• 3 years ago

Great point in the article. In https://opencoreventures.com/blog/2023-06-27-ai-weights-are-... I propose a framework to solve the confusion. From the post: "AI licensing is extremely complex. Unlike software licensing, AI isn’t as simple as applying current proprietary/open source software licenses. AI has multiple components—the source code, weights, data, etc.—that are licensed differently. AI also poses socio-ethical consequences that don’t exist on the same scale as computer software, necessitating more restrictions like behavioral use restrictions, in some cases, and distribution restrictions. Because of these complexities, AI licensing has many layers, including multiple components and additional licensing considerations."

PreInternet01• 3 years ago

It's not just in the LLM space; even for 'older' models, companies have aggressively embraced this approach. For example: YOLOv3 has been appropriated by a company called Ultralytics, which has subsequently released the 'YOLOv5' and 'YOLOv8' "updates": https://github.com/ultralytics/ultralytics

There is no marked increase in model effectiveness in these 'new' versions, but even if you just use the 'YOLOv8' Pytorch weights (and no part of their Python toolchain, which might have some improvements), these will somehow try to download files from Ultralytics servers. Possibly for a good reason, but most likely to, let's say, "pull an Oracle."

Serious AI researchers won't go anywhere near this stuff, but the number of students-slash-potential-interns with "but it's on GitHub!" expectations that I had to reject lately due to "nope, we're not paying these guys for their Enterprise license just to check out your project" is rather disheartening...

rvz• 3 years ago

Yes (Unfortunately). But Llama 2 being released for free as a downloadable AI model is much better than nothing. For now it is a great start against the cloud-only AI models.

As for terms, we'll settle on '$0 downloadable AI models' which are available today. Would rather use that over cloud-only AI models which can fall over and break your app at any time and you have zero control over that.

Stable Diffusion is a good example that fits the definition of 'open-source AI' as we have the entire training data, weights reproduciblity, etc and Llama 2 does not.

FanaHOVA5• 3 years ago

Agreed. I called it a "$3M of FLOPS donation" by Meta.

Havoc• 3 years ago

It is quite an unfortunate dilution of the term

godelski2• 3 years ago

The headline is editorialized. Actual is "LLaMA2 isn't "Open Source" - and why it doesn't matter"

It is actually editorialized in a way that feels quite different from the actual one. I think the author and the poster might disagree on what open source means.

swyx• 3 years ago

they are the same person :)

FanaHOVA5• 3 years ago

Mods changed the title, I used the original one when first posting. Not sure why they changed it.

godelski2• 3 years ago

Maybe Dang is taking a hard stance on the "open source" position. I'm honestly with you, as long as the source is available people will call it open source and complaining over terms isn't going to convince anyone.

indus• 3 years ago

No wonder there is such “momentum” on watermarking.

danShumway10• 3 years ago

For the foreseeable future, open source and open weights will be used interchangeably, and I think that’s okay.

This is a little weird given that directly above, the author puts LLaMA into the "restricted weights" category. Even by the definition the author proposes, LLaMA 2.0 isn't open source; we shouldn't be calling it open source.

If open source in the LLM world means "you can get the weights" and doesn't imply anything about restrictions on their usage, then I don't think that's adapting terminology to a new context, I think it's really cheapening the meaning of Open Source. If you want to refer to specifically "open weights" as open source, I'm a bit more sympathetic to that (although I don't think it's the right terminology to use). But I see where people are coming from -- I'm not too put off by people using open source to describe weights you can download without restrictions on usage.

But LLaMA is not open weights. It's a closed, proprietary set of weights[0] that at best could be compared to source available software.

It is deceptive for Facebook to call LLaMA open source, and we shouldn't go along with that narrative.

[0]: to the extent weights can be copyrighted at all, which I would argue they can't be copyrighted, but that's another conversation.

FanaHOVA5• 3 years ago

Author here. I agree with you. LLaMA2 isn't open source (as my title says, the HN one was modified). My point is that the average person will still call it "open source" because they don't know any better, and it's hard to fix that. Rather than just saying "this isn't open source", we should try to come up with better terminology.

Also, while weights usage might be restricted, it's a very big compute investment shared with the public. They use a 285:1 training tokens to params ratio, and the loss graphs show the model wasn't yet saturated. This is valuable information for other teams looking to train their own models.

LLaMA1 was highly restrictive, but the data mix mentioned in the paper led to the creation of RedPajama, which was used in the training of MPT. There's still plenty of value in this work that will flow to open source, even if it doesn't fit in the traditional labels.

danShumway10• 3 years ago

Thanks for replying! And agreed on the title change; I think your original title is much, much better phrased and I'm sorry that I glossed over it when reading the article (although I'm not sure "doesn't matter" fully captures the distinction you're making here) -- mods probably shouldn't have changed it.

There's still plenty of value in this work that will flow to open source, even if it doesn't fit in the traditional labels.

That is a good point; the fight over what is open source and what is source available can get heated, and part of that is a defense against the erosion of the term. But... in general source available is better than closed source software. And LLaMA 2 is a significant improvement over LLaMA 1 in that regard, it really is. So I don't necessarily want to be down on it, in some ways it's just backlash of being tired of companies stretching definitions. But they're doing a thing that will absolutely help improve open access to LLMs.

I'm always a little bit torn about how to go about this kind of criticism of terminology, and I'm not trying to say that people shouldn't be excited about LLaMA 2. But the way it works out I'm often playing word police because the erosion of the term does make it harder to refer to models with actual open weights like StableLM. Facebook deserves real praise for releasing a model with weights that can be used commercially. It doesn't deserve to be treated as if what it's doing is equivalent to what StabilityAI or RedPanda is doing.

I do like your terminology of "open weights" and "restricted weights", and I wouldn't be opposed to even breaking that down even further, I think there's a clear difference between LLaMA 1 and 2 in terms of user freedom, so I'm not opposed to people trying to distinguish, just... it's not hitting the bar of being open weights.

It's a bit like if the word vegetarian didn't exist, and if everyone argued about how it's unhelpful to say that drinking milk isn't vegan because it's still tangibly different from eating meat. On one hand I agree, but on the other hand it's better to have another category for it that means "not vegan, but still not eating meat." There is an actual danger in blurring a line so much that the line doesn't mean anything anymore, and where people who mean something more rigorous no longer have a term to communicate amongst themselves. If average people get bothered by throwing LLaMA 2 into the "restricted weights" category, it's better to introduce another category between restricted and open that means "restricted but not commercially".

Beyond that though... yeah, I agree. I don't really have a problem with people calling open weights open source, my only objection to that is kind of technical and pedantic, but I don't think it causes any actual harm if someone wants to call StableLM open source.

StackOverlord• 3 years ago

As I said last week, compiling source-code does not cost millions of dollars. How much does it cost to gather training data ? Training llama costed around 30 millions in infrastructure + 50k in power costs (source: https://news.ycombinator.com/item?id=35008694).

arikanev• 3 years ago

How is it possible that you can fine tune Llama v2 but the weights are not available? That doesn’t make sense to me.

Der_Einzige4• 3 years ago

Given that it's basically impossible to prove that a particular text was generated using a particular LLM (and yes, even with all the watermarking tricks we know of, this is and will still be the case), they might as well be interchangeable. Folks can and will simply ignore the silly license BS that the creators put on the LLM.

I hope that users aggressively ignore these restrictive licenses and give the middle finger to greedy companies like Facebook who try to restrict usage of their models. Information deserves to be free, and Aaron Swartz was a saint.

heyzk• 3 years ago

You see a similar loosening of the term in other fields e.g. open source journalism. Although that seems to be more about crowdsourcing than transparency or usage rights.

blueblimp• 3 years ago

What's problematic is that there are big models that adopt truly open source licenses, such as MPT-30b and Falcon-40b. As grateful as I am for having access to the Llama2 weights, it feels unfair that it gets credit for being "open source" when there are competing models that really are open source, in the traditional OSI sense.

The practical difference between the licenses is small enough that I expect most people (including me) will choose Llama2 anyway, because the models are higher quality. But that incentive may mean that we get stuck with these awkward pseudo-open licenses.

throwuwu• 3 years ago

Should be good motivation to figure out what those numbers mean

paulmd• 3 years ago

While it’s mostly open, there are caveats such as you can’t use the model commercially if you had more than 700M MAUs as of the release date, and you also cannot use the model output to train another large language model. These types of restrictions don’t play well with the open source ethos

No, CC-NC-ND is a thing, and even GPL applies restrictions on derivation as well.

"Open source" doesn't mean BSD/MIT. There is even open-source that you cannot freely redistribute at all - not all open-source is FOSS!

I always think it's a testament to how much copyleft has succeeded that in many cases people think of GPL and BSD/MIT as being the baseline.

morpheuskafka• 3 years ago

The chart in this this article is very wrong to show only GPL as free software and MIT/Apache as open source but not free software licenses.

While the FSF side of things doesn't like the term "open source," even they say that "nearly all open source software is free software." Specifically, the MIT and Apache (and LGPL) licenses are absolutely free software licenses--otherwise Debian, FSF-approved distros, etc. would have far less software to choose from.

What the chart probably meant to distinguish is copyleft vs free software or open source. And if you're ordering it from a permissiveness viewpoint, the subset relationship should be reversed--GPL is far more permissive than SSPL, etc., but still less permissive that MIT/Apache.

seba_dos1• 3 years ago

Yup. The terms "Open Source" and "Free Software" are pretty much interchangeable when it comes to licenses. The difference is political, not technical.

This part of the article is pretty misleading as well:

Free software, as specified by the Free Software Foundation, is only a subset of open source software and uses very permissive licenses such as GPL and Apache.

bloppe• 3 years ago

Why not just "downloadable"? It describes the actual difference between LLaMA and GPT. Open-data is the only other distinction that matters.

lazzlazzlazz• 3 years ago

The spirit of "open source" implies "open weights", without doubt. Litigating the specific meaning of the terms is pointless.

Terr_• 3 years ago

downloadable weights

When it comes to "how much of it has to be available to be open source", I think it may be instructive to look at encryption algorithms.

Many of them have numeric constants or initial values--NOT part of the secret key itself--which need to be known and available, both for interoperability and for the expected security-level of the algorithm. These are arguably similar to LLM weights. (Perhaps the simplest example would be the prominence of "13" in ROT13.)

Yet if someone tried saying that their encryption standard was "open source" while keeping those constants secret and/or legally-encumbered, I think a lot of people would complain that the label is incorrect or inappropriate.

pabs3• 3 years ago

I like Debian's ML policy about this:

https://salsa.debian.org/deeplearning-team/ml-policy/

account423• 3 years ago

The author clearly doesn't understand the term open source as it is used for software in the first place as evidenced by the completely nonesensical diagram [0]. And no, the term doesn't need to evolve even if parasites want to in order to ride of the goodwill fostered by the open source community.

[0] https://www.alessiofanelli.com/images/open-models.png

In the LLM space, "open source" is being used to mean "downloadable weights"

Related Stories