Why Meta’s latest large language model survived only three days online

https://www.technologyreview.com/2022/11/18/1063487/meta-large-language-model-ai-only-survived-three-days-gpt-3-science/

by p0pcult • 4 years ago

19 7 4 years ago

technologyreview.com

selfmodruntime• 4 years ago

Stories like these are the reason why I immediately download and backup software if I even get a tiny inkling that it may be „too free“.

Philadelphia• 4 years ago

“A fundamental problem with Galactica is that it is not able to distinguish truth from falsehood, a basic requirement for a language model designed to generate scientific text. People found that it made up fake papers (sometimes attributing them to real authors), and generated wiki articles about the history of bears in space as readily as ones about protein complexes and the speed of light.”

How would anyone think releasing this was a good idea?

vba616• 4 years ago

People have been expressing surprise about this issue with computing machinery for 200 years.

mfld• 4 years ago

1. There is a large corpus of scientific publications.

2. Thanks to scihub et al, it is possible to download them en masse.

3. It is now feasible to train good language models on a large text corpus.

4. Wouldn't it be interesting to see what a model trained on this data comes up with? Maybe even new connections in existing knowledge?

neweroldguy• 4 years ago

How would anyone think releasing this was a good idea?

Why would it not be? A model built on any consistent large data set could be interesting I’d expect.

But just because the data set has some semantic property, that doesn’t mean that a model built from it will necessarily generate text that had the same property (in this case, “being scientifically correct”). The real question is why would anyone think that?

p0pcult• 4 years ago

Good points on both sides. I think the answer to your second question is, "first mover advantage, even if the product is bad"

bpanon• 4 years ago

Humans do the dame thing.

Why Meta’s latest large language model survived only three days online

Related Stories