Danger, AI Scientist, Danger

https://thezvi.substack.com/p/danger-ai-scientist-danger

by ben_w • 2 years ago

29 17 2 years ago

thezvi.substack.com

jksk61• 2 years ago

funny paper, I still don't know what was the goal of it. It is evident to anyone that LLM can't perform any meaningful reasoning, why even bothering in building such an infrastructure to test whether it is able to become a "scientist".

kkzz993• 2 years ago

It is evident to anyone that LLM can't perform any meaningful reasoning

Its not and its pretty evident to anyone that has actually used SotA LLMs for more than 5 minutes.

jjtheblunt2• 2 years ago

Are there any published scientific studies indicating such? If so, i have missed them so far.

somenameforme• 2 years ago

They do a phenomenal job of guessing the next word, and our language is redundant enough that that alone, carried out recursively, can produce quite interesting results. But reasoning? I'm certain everybody has gotten in this pattern, because it happens on pretty much anything where the LLM doesn't answer right on the first shot:

---

LLM: The answer is A.

Me: That's wrong. Try again.

LLM: Oh I'm sorry, you're completely right. The answer is B.

Me: That's wrong. Try again.

LLM: Oh I'm sorry, you're completely right. The answer is A.

Me: Time to short NVDA.

LLM: As an AI language learning model without real-time market data or the ability to predict future stock movements, I can't advise on whether it's an appropriate time to short NVIDIA or any other stock.

---

imtringued2• 2 years ago

Whenever you poke people about LLMs solving decidable/computable problems, they get defensive and claim that they are not good for that. You are supposed to generate code that solves the decidable problem, heavily implying that retrieval, approximation and translation are the only true capabilities of LLMs.

throwup238• 2 years ago

There is zero chance anyone can rigorously evaluate reasoning ability in five minutes.

rcxdude• 2 years ago

I think it depends on your standards. LLMs are by far the best general purpose artificial reasoning system we've made yet, but also they aren't really very good at it. Especially more complex steps and things that require rigor (chain-of-thought prompting and such helps, but still, they have super-human knowledge but the reasoning skills of maybe a young child)

FormerCoolGuy• 2 years ago

It's a Tokyo based company, Sakana means fish in Japanese, and the icon of the company is a bunch of fish. What's up with the Danger AI Labs Hebrew translation in the tweet and article?

TheBozzCL• 2 years ago

Because it means “danger” in Hebrew: https://pollylingu.al/he/en/items/30106

And the author thought it was funny

Danger, AI Scientist, Danger

Related Stories