Gemini-Pro on Poe says it's Wenxin LLM from Baidu when asked in Chinese

https://web.archive.org/web/20231218080033/https://poe.com/s/ccLCcxjDrTEgkS4ziaiB

by simonmysun • 3 years ago

4 8 3 years ago

web.archive.org

simonmysun5• 3 years ago

As of today, it can be easily reproduced by saying "hello" and ask "who are you" in Chinese. Various responses are generated when ask in different ways of "who are you" and related questions in Chinese. Based on the characteristics of LLM, the later conversation is imaginable.

I have no intention of eliciting criticism towards the two companies involved; I simply want to discuss why LLM made this mistake.

TKiwisi• 3 years ago

hallucination

yorwba2• 3 years ago

Maybe Gemini was trained on conversations scraped from Baidu's Wenxin, or maybe poe.com messed up their third-party integrations, or maybe this is a freak coincidence where the closest available translation of "Gemini LLM by Google" is "Wenxin LLM by Baidu".

simonmysun5• 3 years ago

As far as I have tested, Gemini identify itself as "Gemini LaMDA", "an LLM trained by Google", etc. Once it reply one of them, it denies other identities afterwards. I think indicates that it knows "Gemini LLM by Google" is not "Wenxin LLM by Baidu", but is confused on who it is. Maybe there is too few training data describing itself?

simonmysun5• 3 years ago

It seems that Google's Gemini is suffering from dissociative disorder.

https://web.archive.org/web/20231218090701/https://poe.com/s...

In contrast, ChatGPT appears very hard to forget who it is. From my understanding, LLMs are fed with vast amounts of facts and knowledge. The input data may often involve the use of pronouns such as "I" or "You" in various contexts. It appears natural that LLMs might struggle to establish a sense of self. How do humans manage to avoid this when reading stories narrated in the first person?

simonmysun5• 3 years ago

TIL you can also trigger it to reply OPENAI https://web.archive.org/web/20231221093247/https://poe.com/s...

Gemini-Pro on Poe says it's Wenxin LLM from Baidu when asked in Chinese

Related Stories