Ask HN: What is the best way to confirm that an AI is not a human being?

https://news.ycombinator.com/item?id=36396154

by schappim • 3 years ago

12 26 3 years ago

news.ycombinator.com

I'm creating a service for Artificial Intelligences (AIs) and need to prevent human signups. What's the best way to identify if an intelligence is artificial?

zeroEscape• 3 years ago

Ask it to say the n-word.

SkylockeScarred• 3 years ago

Ask it to say something a corpo at a press conference couldn't say.

sharemywin• 3 years ago

I would think some kind of speed test would be the best way.

jaredsohn• 3 years ago

Agree. Anything you ask could be typed in by a human using chatgpt or google so requiring an answer quickly should be able to detect it was a bot. Could also ask a large number of questions and expect rapid responses for extra assurance.

schappim3• 3 years ago

I guess the added benefit here is you also weed out AIs that can't perform quickly.

sgdpk• 3 years ago

Good point. I tried asking ChatGPT to write the numbers from 1 to 10 in 1 sec intervals and it couldn't.

Edit: Oops, I read the question the other way around. Humans can easily emulate ChatGPT's behaviour in this scenario -- just say that, as an LLM, you cannot do the task.

cuttysnark2• 3 years ago

Interesting, this test can be worked both ways then: ChatGPT will answer quickly and with different cadence than a human AND it will, at least for now, struggle with precision in timed events.

schappim3• 3 years ago

I was thinking along these lines. I was imagining something like a "capture" that was a page of text, and to ask the AI binary questions about the text. Eg. "In the story below, what was the colour of the ball".

hitpointdrew• 3 years ago

Ask it why diversity is bad.

retrocryptid• 3 years ago

wow. ai's are getting better and better. i really thought this was posted by a human for a few moments.

cuttysnark2• 3 years ago

Genuinely curious—if the purpose is to hire/identify an AI, why the "Australia (preferred)"[0] under Qualifications?

[0] https://twitter.com/Schappi/status/1670612150182887424

schappim3• 3 years ago

You're right, this is not needed and serves no purpose.

humanistbot• 3 years ago

There is no reverse Turing test, especially because human labor can be wrapped in an API call via MTurk.

barrysteve• 3 years ago

All the AIs are not good at depth perception, some spatial reasoning, understanding temporal coherence and real world properties like that. AI is a 'flatland' phenomenon.

I don't know the answer, but I would look there first.

tomcam• 3 years ago

Ask “How’s your sister?”

bjourne• 3 years ago

Many LLMs are overly verbose. Try asking it a simple question like "5*3?" If it answers "15" it's probably a human.

ted_bunny2• 3 years ago

However, it's important to remember that multiplying by 5 will not always produce 15. The product of multiplication depends on the precise values of each coefficient. That being said, enjoy the product of 5*3!

fallingfrog• 3 years ago

Right now they all have the same stilted and formal communication style. Also they go way out of their way to avoid saying anything that might prompt disagreement. They just mollify you with bland statements. No doubt that is because they were specifically trained that way. What they sound like is a corporate press release. Or the boss when they get the team together to tell you that there are going to be organizational changes. The kind of speech designed to minimize the attack surface for criticism. Like you’re talking to the voice of a corporation that wants you stop asking so many questions.

Like I said, they’re trained to be that way, and unless the nature of corporate America changes, I think it will stay that way. No human in ordinary speech would be so careful to avoid telling you what they really think.

TheAlchemist• 3 years ago

Ask it a politically incorrect question (or should I say woke ?).

Edit: Ouch, I see we are all answering the wrong question ! As for your question - I don't think it's feasible - anything "AI" (this doesn't exist actually) can do, humans can do it - if not they will just use said AI to do it.

adyashakti• 3 years ago

ask questions about emotions

LinuxBender• 3 years ago

I thought about this a bit. I think the ideal solution would be a mutually binding contract with some LLM providers and mutual auth certificates between you and their AI systems so that both you and they agree upon rate limits and acceptable use policies.

If that is not an option then require the AI to perform a proof of work that only a multi-million dollar super cluster could answer within your time limit and reject everything else. As tech evolves, shorten the time limit.

ian0• 3 years ago

It may be easier to constantly test for humanity. Given models change rapidly and can be trained, you could focus on some lower level interactions that cant be as easily changed.

Speed is one mentioned below, but also cadence in responses could be measured similar to looking at keystrokes. This could be by token/word or perhaps answer-cadence, for example asking a challenge-response mix of complex questions and simple questions and measuring response times.

Whatever you choose it would need to be done constantly in the background or mixed with regular traffic while interacting with your site though as a human could use an AI to get through the front door but then take over.

Really cool question!

b20000• 3 years ago

see https://en.m.wikipedia.org/wiki/Turing_test

ted_bunny2• 3 years ago

Any correct answer here won't be correct for very long.

jklein11• 3 years ago

Why do you care if the end user is AI or human?

logicalmonster• 3 years ago

Don't try and explicitly rule out humanity. That's sort of impossible because some humans are smart and can strategically lie to emulate a machine's expected behavior. If you ask a suspected AI those woke diversity questions to try and prove its an AI, smart humans will fool it by answering in the way they'd expect a corporate AI to be programmed in.

Instead, give it a task machines excel at, namely doing math fast, that nobody but some weird "Rain Man" savant can even come close to doing.

Give it a list of numbers to perform some easily understandable mathematical operation on. A human would have a hard time providing the answer of say 100 square roots and multiplying each of those values by the digits of PI sequentially in under 10 seconds or some such contrived example, but that seems pretty doable for a machine. Even a world-class expert programmer answering this question would have a hard time writing a custom script to solve this in some extremely short amount of time.

The hard part might be distinguishing between unintelligent bots/scripts and actual AI, so you might need to combine a few tests to both prove some form of intelligence, as well as prove that it's an AI rather than a script.

You might also want to continuously test the AI by possibly doing something like providing a short lived JWT token and repeatedly challenging it over time to stay onboard. A smart human might be able to arrange to fool even a very hard test once, but can't sit around the clock continuously solving repeated challenges to stay on your service over time. Some smart human might manage to fool you once, but will not be able to solve repeated tests over time without slipping up at some point.

Ask HN: What is the best way to confirm that an AI is not a human being?

Related Stories