Claude Sonnet and Pi AI give exact same response to a prompt

https://twitter.com/seshubon/status/1765870717844050221

by rmorey • 2 years ago

14 6 2 years ago

dpflan• 2 years ago

Does someone have more context as why this is so “insane”? Yes understandably there is randomness / non-deterministic nature to these generative models. But what does this mean to you as user? What’s so compelling about this particular scenario? Maybe you discover that some apps are just wrappers around another API.

What’re some other angles to this that make this a good post for discussion?

jacooper• 2 years ago

I think because inflectionai should be using their own in-house models.

basil-rash• 2 years ago

The insane part is how we’ve been conditioned to interpret the most banal, 0-entropy strings as “cutting edge AI”, to the extent we get surprised when two supposedly different models emit the same (lack of) content.

Seriously, look at the response. It’s the exact sort of thing the top 150 blogspam results on Google would all say if you were to ask the same question.

rmorey• 2 years ago

The models behind these products, Claude 3 Sonnet, and Inflection-2.5 are ostensibly completely separate models, trained independently. It is hard enough to get the same model to produce the same result twice at temp>0, so it’s pretty suprising that two separate models would give the same result word for word, at such length!

singularity2001• 2 years ago

That's even doubly strange since even identical models should emit different answers (unless temperature is set to 0 which is almost never default?)

luke-stanley• 2 years ago

Most likely it is a Claude Sonnet model being used for that conversation in a split test.

Claude Sonnet and Pi AI give exact same response to a prompt

Related Stories