Telling AI model to “take a deep breath” causes math scores to soar in study

https://arstechnica.com/information-technology/2023/09/telling-ai-model-to-take-a-deep-breath-causes-math-scores-to-soar-in-study/

by peterbonney • 3 years ago

8 4 3 years ago

arstechnica.com

patleeman• 3 years ago

The actual prompt is just a chain of thought prompt and the headline is just clickbait.

Also in the article, the author gets COT wrong.

What "reasoning" they do (and "reasoning" is a contentious term among some, though it is readily used as a term of art in AI) is borrowed from a massive data set of language phrases scraped from books and the web. That includes things like Q&A forums, which include many examples of "let's take a deep breath" or "think step by step" before showing more carefully reasoned solutions. Those phrases may help the LLM tap into better answers or produce better examples of reasoning or solving problems from the data set it absorbed into its neural network weights.

Chain of thought has nothing to do with “tapping into better answers”. It’s simply asking the model to break up the output into smaller tasks and gives it more time and space to reason.

COT is not new or novel. Hell, it’s even listed in one of the guides in Open AI’s prompt guides as a strategy to improve prompts.

dragonwriter2• 3 years ago

Chain of thought has nothing to do with “tapping into better answers”. It’s simply asking the model to break up the output into smaller tasks and gives it more time and space to reason.

It doesn't give it more time and space to reason, though. Time isn't usually bounded on a single turn and space is context-window limited (and every turn in Chain of Thought is done within the context window, so it doesn't add any space.)

What it does is push the output toward a shape that resembles a particular idealization of (the explanation of) human reasoning, producing results that look more like an explanation of reasoning and sometimes producing more satisfying conclusions.

Tiberium• 3 years ago

It does give more space in the sense that the model generates more tokens of steps/etc that it can then base its actual answer on, rather than being forced into generating the answer right away.

Telling AI model to “take a deep breath” causes math scores to soar in study

Related Stories