Can LLMs Generate Novel Research Ideas?

https://www.arxiv.org/abs/2409.04109

by kmdupree • 2 years ago

50 80 2 years ago

Even if they can't, individuals are prone to inspiration from an LLMs attempt. Worth giving it a shot at least.

ModernMech• 2 years ago

This is true, I find LLMs to be the best mental lubricant ever created. Cures writers block and gets me out of creative and technical jams all the time.

marmaduke3• 2 years ago

Devils advocate, writers block and jams are symptoms not bugs to fix

dageshi• 2 years ago

I'm outlining a fantasy story I want to write, I've found using image generation (leonardo.ai) very helpful in nailing down what the world looks like and how it'll work, I've actually changed my mind on a few things based on what the AI generated.

Plus honestly it's just genuinely fun! I've resisted moving to one of their paid packages just yet because I think I'd spend all day on it!

kelseyfrog• 2 years ago

Even if we've essentially created a rubber duck that slightly more easy to engage with, that sounds like a win.

vunderba• 2 years ago

Strongly agree.

Back in high school, I put together a program called DreamPool that would just literally pick out a few nouns from a gigantic dictionary file and bubble them up in a little graphic of a well, and then I would sit quietly and spin those concepts around attempting to connect them together.

LLMs are like a version of this on steroids and the potential as a tool for augmentation is huge.

influx• 2 years ago

Dreampool sounds very cool, almost like brain excercise!

amelius2• 2 years ago

No.

Perhaps we can use LLMs to invalidate patents. Want to check a patent? Just download the LLM model from before the issue date, then ask the LLM to produce the work. If it succeeds, you have invalidated the patent because the work was not novel.

bee_rider• 2 years ago

Huh.

Regardless of folks opinions on the actual plan, does this idea imply an interesting question?

In some sense all of the ideas in a book already exist. Before I read the book, they haven’t been put through a process of being interpreted by my eyeballs and ingested into my brain. But, they do already exist.

Do the ideas in an LLM latent space (or whatever) already exist? They haven’t been read or interpreted yet, but the sentient cognition that goes into creating the idea has already happened.

What does it mean for an idea to exist anyway?

ninetyninenine5• 2 years ago

https://openai.com/index/learning-to-reason-with-llms/

amelius2• 2 years ago

Conversely, you may ask: do we really want to grant patents for ideas that can be generated simply by asking an llm?

jedberg3• 2 years ago

An LLM is like a well read college student with a nearly photographic memory that sometimes mixes things up.

It's great for bouncing ideas off of and getting feedback on them. And yeah, it might product "novel ideas" by mixing and matching existing ideas, but LLMs will never create truly novel ideas. Not in their current form.

The paper didn't really answer the question sadly: their conclusion was just that humans rate LLM answers as more novel than human ones, but less feasible.

Zambyte• 2 years ago

And yeah, it might product "novel ideas" by mixing and matching existing ideas, but LLMs will never create truly novel ideas. Not in their current form.

Can you give a historic example of a human creating "truly novel ideas" that is not the product of mixing and matching existing ideas?

jedberg3• 2 years ago

I mean, it depends on how pedantic you want to be. I'd say e=mc2 was pretty novel, but yes, it was based on existing math concepts.

But I don't think an LLM could ever come up with something like that.

api• 2 years ago

This argument can be applied recursively. For example: the information in the human genome is just information from the environment transferred into the genome by evolutionary learning. Ultimately you get back to some kind of first cause argument where everything goes back to God or whatever natural process created information in the universe. Either nothing new is actually new or everything new is new.

In the end it becomes moot. A novel rearrangement of existing ideas that does something new or different is creativity.

Can LLMs do that? I think they can to a limited extent, but not as well as humans. Is it something we get with scale or does it require a fundamental architectural innovation? Don't know.

F-Lexx• 2 years ago

Can you give a historic example of a human creating "truly novel ideas" that is not the product of mixing and matching existing ideas?

The invention of PCR comes to mind:

During a symposium held for centenarian Albert Hofmann, Hofmann said Mullis had told him that LSD had "helped him develop the polymerase chain reaction that helps amplify specific DNA sequences".

https://en.m.wikipedia.org/wiki/Kary_Mullis

jedberg3• 2 years ago

How about the Transformer models that power LLMs? That was pretty novel.

PaulHoule2• 2 years ago

Creativity is one of the most problematic concepts in psychology. I am right now reading

https://www.amazon.com/Sounds-Bell-Jar-Psychotic-Authors/dp/...

where the authors (a psychologist and two literature critics) carefully tease apart the connection between creativity and psychosis which is of course problematic because insanity mostly gets in the way of being creative which leads to much more serious definition of what "being creative" really means than one usually finds. (One thing they point out is that a third-rate artist (Andy Warhol?) can become quite prominent if they are good at marketing their work.)

People who are religious will make a theological argument to the effect that "God gave you the power to create when he created you" or "You can be creative because God is inside you".

Atheists may dismiss these arguments out of hand but it's a mistake to do so because of

https://en.wikipedia.org/wiki/Ontological_argument

in the sense that "God" can be defined as "the reason why there is something instead of nothing" which could have no relation to the image of some old patriarch on a throne. If we are made "in his image" we should consider the image revealed in a microscope that reveals that we are based on cybernetic principles that apply to the individual cell as well the whole organism and how those principles apply to the evolution of language and culture as they do to our genetic endowment.

(Insofar as God can delegate his creative ability to you, can't you further delegate it?)

A vulgar version of this is Rodger Penrose's "I can solve math problems because I am a thetan" where he claims to be exempt from the problems that Godel and Tarski and Turing warned you about but since there is nothing complete or consistent about Rodger Penrose these don’t apply (he can't solve Collatz and neither can a OT VIII!)

Muddy thinkers may reject the existence or relevance of God or not explicitly believe they are "a spirit in the material world" but often think there is something uniquely human about creativity (can other animals be creative?) but I sense that the ghost of the arguments above is behind that thinking.

In this transcript I get Python to create something that was never seen before and will never be seen again

   >>> uuid4()
   UUID('21205a92-2611-4710-b120-4a94f5ccf2d9')

which is by no means interesting; real creativity involves creating something that is useful and/or expressive using certain resources and subject to some system of constraints. Insofar as some task is repeatable, creativity is involved in the creation of some process or and/or system that makes the task repeatable.

As Edison put it “Genius is one percent inspiration and ninety-nine percent perspiration” so it is not so interesting that the LLM can generate novel (yuck I hate that word, "novel" is the first word I delete when I have to squash a long paper title to fit into 80 characters) research ideas, I'll be impressed when it can fill out a grant application that gets funded.

ryandvm• 2 years ago

Let's be real though. 99% of people 99% of the time are also not coming up with novel ideas.

LLMs seem to mostly be limited right now by the fact that they're always losing context on new conversations and their interactions don't rewire their neural nets. Hard to come up with a new idea when your brain gets reset with every conversation.

owenpalmer• 2 years ago

Hard to come up with a new idea when your brain gets reset with every conversation.

Haha I love this picture!

chaosist• 2 years ago

I agree but I have tried many times to intersect two ideas with a LLM that would be novel and the LLM can not do this at all.

We shouldn't expect the stochastic parrot to be able to do this though and it is unfair to the stochastic parrot.

It is like expecting a real parrot to say words it has never heard before.

No one asks that of a real parrot because we don't anthropomorphize a real parrot like we do the LLM.

lostmsu2• 2 years ago

You are trivially incorrect.

LLMs (unless used in deterministic mode, which you shouldn't anyway) will eventually generate all ideas for the same reason as 1_000_000 monkeys on typewriters will eventually generate War and Peace. The question is only how soon in practice this will happen.

Well, I am nearly certain, that 1_000_000 LLMs (or rather 1_000_000 streams of generation using a few LLMs) will do better than 1_000_000 monkeys on typewriters.

The same 1_000_000 LLMs could do better than 1_000_000 average humans, but we don't know yet.

breck2• 2 years ago

Yes.

LLMs have helped me generate 1 novel discovery: that every top 10 programming language has a single creator (https://pldb.io/blog/aSingleCreator.html).

They also helped me generate this map yesterday (https://pldb.io/blog/whereInnovation.html), which is the most comprehensive map of programming language creation and software innovation ever created.

svieira• 2 years ago

Apache is credited with things it didn't initiate, but accepted after they were given up by another company (e. g. Cassandra)

smokel• 2 years ago

How do you define single creator? The person who typed the first keystroke?

It seems pretty silly to claim that Algol 60 has thirteen creators. Sure, Wikipedia lists that many names in some table, but that is not really a significant statistic, now is it? Java 22 probably has hundreds of people who contributed to it.

Fun exercise though :)

breck2• 2 years ago

How do you define single creator?

The creators define how many creators there were.

It's all open source and anyone can fix any mistakes by updating one line (for example, here's the creators entry for Python: https://github.com/breck7/pldb/blob/b8ae74253733e4aa0fb57d26...).

We generally don't have any disputes but there's the occasional error and always open to pull requests to fix those.

But you do bring up a good point about contributors/maintainers. Adding that data is definitely on the priority list, but might be a 2025/2026 thing.

mercurialsolo• 2 years ago

The difference between novelty and hallucination is feasibility. With integrated critique and feasibility checks you can eventually map the hallucination space into novelty.

uludag• 2 years ago

I'd be curious what an LLM would be able to accomplish if it was given all pre-modern texts we have access to. Would it be able to come up with ideas we know as modern (political theories, philosophical ideas, scientific theories, etc.)?

I have a hard time believing that if we fed an LLM all prehistoric speech uttered from humans that no matter what, it would never escape the paradigms of those people. If that is the case, than would relying on LLMs just get us stuck in our own paradigms and prevent true "progress"?

Teever3• 2 years ago

I've wondered that too. If we trained an LLM on only the scientific content that predates Einstein could it come up with general relativity?

How much coaxing would it require to get it there? What would that process be like?

Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process?

staticman23• 2 years ago

> Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process?

The parts of an LLM that teach it language is not disconnected from the parts that teach it facts. Good luck teaching it language with only pre Einstein data.

deisteve• 2 years ago

i just read this paper on arxiv and im still trying to wrap my head around the implications so the authors got over 100 nlp researchers to write down novel research ideas and then had them review ideas generated by a large language model (llm) without knowing which ones were human generated and which ones were from the llm and the results are pretty fascinating the llm generated ideas were actually judged to be more novel than the human ones p value < 0.05 but slightly weaker in terms of feasibility i mean whats the point of having a novel idea if its not feasible right but still its pretty cool that the llm can come up with stuff that humans havent thought of before

and the authors are saying that this study highlights some of the open problems in building research agents that can generate novel ideas like the llm was bad at self evaluation it couldnt tell which of its own ideas were good or bad and they also found that the llm generated ideas that were too similar to each other lacking diversity i mean thats not surprising right llms are trained on huge datasets but theyre still just pattern recognition machines they dont really understand the context or the implications of what theyre generating

but heres the thing novelty is hard to judge even for experts i mean how do you even define novelty is it just something that nobody has thought of before or is it something that challenges our current understanding of the world and the authors are proposing a follow up study where they actually have researchers execute these ideas into full projects to see if the novelty and feasibility judgements actually translate into meaningful differences in research outcomes which is a great idea i mean thats the only way we can really know if these llms are useful for accelerating scientific discovery or not

anyway im rambling on now but i just think this is a really interesting area of research and im excited to see where it goes can we really use llms to accelerate scientific discovery and what are the limitations of these models and how can we overcome them etc etc

karmakurtisaani• 2 years ago

Your output would benefit from proper punctuation. Are you using some kind of speech-to-text application? Certainly looks like it.

Interesting comment nevertheless, if not somewhat difficult to parse.

onemoresoop• 2 years ago

LLM are just large language models, they're not capable of thinking or coming up with novel ideas. They could enable people to do so by providing food for brainstorming.

mkaic• 2 years ago

"...they're not capable of thinking or coming up with novel ideas."

This claim is unfalsifiable given common definitions of "thinking" and "coming up with novel ideas".

janalsncm• 2 years ago

Anytime I see an argument of the form X is just Y an alarm bell goes off in my head. While it is descriptive of the components, these arguments ignore the possibility of emergent complexity. Sometimes things are more than just the sum of their parts.

The sun is just a bunch of hydrogen.

A computer is just a bunch of transistors.

Humans are just a bunch of cells.

An artificial neural network is just a bunch of matrix multiplications.

I personally think LLMs are extremely limited and overhyped. But this form of argument seems incorrect to me since it can be used to argue that LLMs cannot do things we already know they can do.

mensetmanusman4• 2 years ago

An LLM could come up with an infinite amount of new theories and ideas, of which >99% would be meaningless, if given enough time and energy.

Human’s ability is in guiding the prompt (of AIs, and the mind) on what is worth knowing and which hypotheses are worth testing.

Choosing an action from the countable infinite number of actions is the general framework of free will.

ninetyninenine5• 2 years ago

I think the amazing thing about the LLM is that it's not a random text generator so it's not 99% meaningless.

I give it only 50% meaningless.

eagerpace• 2 years ago

Is this how our simulation started?

fasa99• 2 years ago

It's a brilliant idea, but LLMs are predicated upon the massive data of the internet.

That being said, you could take a pre-2018 dataset and test it for discoveries/insights circa 2019 or 2020 and see how well it performs.

nullc• 2 years ago

shuf -n 3 /usr/dict/words will also sometimes generate novel research ideas.

browningstreet• 2 years ago

Harmony Korine, the poet (done at SXSW 2010 with GordonandtheWhale.com)

https://youtu.be/PqvAmlnJTfU?si=C8M7IGnJ_iYJE-8_

pfannkuchen• 2 years ago

Would you be able to tell if it did? There could be an obscure document in the training set that contains the idea. It seems like a very hard problem to definitively detect whether a concept came from the training set.

valine• 2 years ago

A single document doesn’t contribute much to the loss during base training. LLMs can absolutely memorize text if it’s duplicated in the training set, not so much if the document in question is obscure.

fzeindl• 2 years ago

Could rolling dice with matching phrases generate novel research ideas? (ok the metaphor is slightly off, but not a lot)

AnimalMuppet• 2 years ago

Maybe yes.

If you give it phrases from, say, the last five years' worth of published research papers, and it combines phrases at random and spits out the words, yes, in that there will be some interesting research ideas, and maybe even some that a human would not have come up with.

Unfortunately, they'll be buried in a blizzard of stuff that no human would come up with, because they are totally meaningless combinations. Finding the good ones is the issue - and it's not an issue that LLMs can currently solve.

6510• 2 years ago

For some reason my mind maps concepts from chess onto the real world and the other way around. The novel AI ideas in chess are usually combinations of several... well... bad ideas. People usually take on one bad idea at a time and try to make it work. If they succeed it is quite surprising because objectively it was a bad idea.

Our thinking is a lot less error prone than that of the LLM's but we have to study for years to absorb prior art that LLM's mostly receive at birth by osmosis. It won't be able to take on the truly stupid ideas but it can combine large numbers of the somewhat stupid.

Like a chess position with lots of possible moves followed by lots of possible responses.

Incipient• 2 years ago

Taking a step back. Define "novel".

I have the idea for an induced draft umbrella. Stick a fan at the top under an opening.

Is that idea novel? I haven't seen it anywhere, it's just something I came up with. But it's not entirely novel, I'm just borrowing the concept of a fan, and an umbrella.

I don't feel this is entirely out of scope for what an LLM could describe in words?

jrsharma• 2 years ago

Looks like @sama's o1 is killing PhD science questions. https://x.com/sama/status/1834283100639297910?t=iCRehNoBofMP...

Can LLMs Generate Novel Research Ideas?

Related Stories