When did Neil Armstrong set foot on Mars?

now we can confirm that AI as as flawed as human intelligence itself. It just dynamically reads all the texts and remembers what it wants without being sure it is right.

willchang• 5 years ago

This yields the same result:

  when did neil armstrong set foot on poop

sorokod3• 5 years ago

July 20, 1969 was a busy day for Neil Armstrong. I can easily imagine a children's picture book. Sponsored by Google perhaps.

Y_Y• 5 years ago

I imagine you could even get a couple of GANs to do the illustrations for you.

If nobody has a startup churning out heartwarming books for early childhood using GPT and DCGANs then I call dibs.

rplnt• 5 years ago

Not for me (Mars works)

iosonofuturista• 5 years ago

Cheese also provides amusing results

https://i.postimg.cc/gcwS9BXB/Screenshot-20210818-193325-453...

redleggedfrog• 5 years ago

Happens to the best of us.

unfunco• 5 years ago

Also:

When did Neil Armstrong moon John F. Kennedy

OneLeggedCat• 5 years ago

Nice. Now I'm wondering how far we can take this...

"When did Keith Moon land with Louis Armstrong on Mars"

https://imgur.com/a/TNzjZvm

stevecat• 5 years ago

More likely on Apollo 10 https://www.google.com/amp/s/www.vox.com/platform/amp/2015/5...

ModernMech• 5 years ago

For that matter, so does this:

  when did neil armstrong set foot on earth

notahacker4• 5 years ago

As does "when did Louis Armstrong set foot on the moon" and "when did Richard Nixon set foot on the moon"?

JFK, on the other hand, apparently "set foot" on the moon on May 25 1961 and "land[ed]" on the moon on September 12 1962, and Arthur C Clarke some time in 1968

Gives a good answer for Homer Simpson though

allanrbo• 5 years ago

Randall Munroe, the xkcd guy, has some fun takes on these Google fact-box mistakes: https://twitter.com/xkcd/status/1333529967079120896

tedsanders• 5 years ago

Randall Monroe shared a nice series of these last year: https://twitter.com/xkcd/status/1333529967079120896?s=20

"What year did Tom Hanks land on the moon?" "1970"

"When did the Simpsons learn to control fire?" "Dec 17, 1989"

"Which river did George Washington drown in?" "The Allegheny River"

It's got to be tough to train an AI that can generalize to (or at least recognize) questions outside its training set. Unfortunately, when 99% of questions containing 'when' and 'Neil Armstrong' have the answer 1969, it will need a pretty sophisticated algo to avoid the trap of learning that it should always answer 1969.

I wrote up another example of this last year based on a question with a real answer that Google still gets wrong: https://www.tedsanders.com/why-does-google-think-the-slowest...

Google (and plenty of other search engines) will all helpfully tell you that the slowest animal is the sloth, which is plainly incorrect if you just look at a video of a sloth and then look at a video of a worm. Despite their reputation, sloths still move visibly.

vlovich123• 5 years ago

The last one actually is an example of the ambiguity of language. For example, if you asked this from a person, answers like “sponge”, “mollusk” and “flatworm” probably aren’t what you’re really considering and you might think the other person is being too pedantic. So really you want nuance from the AI like “if you meant all animals, X. If you meant all land animals, Y. If you meant mammals, Z”. Human are very ambiguous with categories because all categories are arbitrary human classifications and dealing with that ambiguity is difficult even for humans.

jcranmer2• 5 years ago

A fun one I tried was "Which president became supreme dictator" that answered "William Howard Taft." It took some finessing to get it (I originally started with "dictator for life"), but it's clear that it picked up only "supreme" and "president" in the query to guess that you meant to ask which president became a Supreme Court justice.

II2II• 5 years ago

The problem isn't so much the answers, but how they are framed.

In the case of Neil Armstrong, the failure is to state that the answer is for a Moon landing. That would allow the person making the query to realize that the answer was not what they were looking for. In this case it is because Armstrong never landed on Mars. In other cases it may be because the question or data were incorrect.

The current Tom Hanks answer is closer to what should have been done in this regard since it refers to the film directly, though it remains problematic in that it highlights the year to such a degree that the person making the query may ignore the context. (It is also problematic because the film is about a mission that didn't land on the Moon, which could only be determined through further research. Granted, that is more along the lines of your slowest animal example.)

kelnos• 5 years ago

Google (and plenty of other search engines) will all helpfully tell you that the slowest animal is the sloth, which is plainly incorrect if you just look at a video of a sloth and then look at a video of a worm.

Would your average person consider a worm an "animal". In the scientific sense, sure, pretty much everything living that is not a plant is an animal, but I think most people don't think of worms when they think of animals.

You can say, "well that's wrong", but that's kinda missing the point. Categories are somewhat arbitrary, and word usage can differ greatly from the scientific or dictionary definition when we're talking about colloquial usage.

repler• 5 years ago

worked for Saturn too

heavymark• 5 years ago

Interesting. When searching for, "when did neil armstrong set foot on venus?" It, shows, "In which year and when Armstrong set his foot on the moon?" and provides the correct year. That's presumably what should show when searching for Mars (or any non earth moon planets).

rafaelturk• 5 years ago

AI will be the next great source fake news, fake stuff. This one was easy to flag as bogus. Problem is that for other topics it will give average user the impression that this is actully a fact. To make it worse Google UI is massively misleading, big bold text, makes it look as legit.

5faulker• 5 years ago

Virtual and physical reality is now one. Good grief.

a_square_peg2• 5 years ago

This is actually a really good example of the limitation of current ML/NLP approaches, that there isn't really any level of 'comprehension' at all.

cryptoz• 5 years ago

isn't really any level of 'comprehension' at all.

I mean, that's a bit harsh. I bet there are lots of people who would answer the question the exact same way. They don't have a total lack of comprehension, they perhaps misheard a word or misremembered a fact they once knew. Honestly, while the Google answer is wrong and this demonstrates a major flaw in their confidence of answering queries, the level of comprehension is still quite impressive (to me, at least)

Xplune134• 5 years ago

While I agree that "no comprehension at all" seems harsh, I don't think anyone would jumble things up between moon and mars unless they got the fact wrong in the first place.

These knowledge cards are pretty useful, but they shouldn't be taken as the source of fact (at least right now) unless one opens the link to check where that card is extracted from.

make3• 5 years ago

The problem is that model is only being asked what the answer is most likely to be, not whether there exists a good answer.

There should be a different model that checks if there is an answer or not, like SQuAD 2.0 https://rajpurkar.github.io/SQuAD-explorer/

bootwoot• 5 years ago

I think the conditional display of the fact box implies they are in fact asking whether there is a good (enough) answer. They are just getting it wrong here.

big_curses• 5 years ago

But at the same time, I think it's doing a really good job of what it's trying to do. Google search is not trying to be a repository for all the world's information. It's just trying to get people to what they're most likely looking for or show the most related things. Given the significance of the moon landing and the fact that no one has set foot on mars I find it unsurprising that it brings up info on the moon landing. It's seems better to assume what the user is likely looking for especially when (at least my) Google searches often take the form of "moon land neil year". I can just type things like that out, stream of consciousness, and the majority of the time I get what I'm looking for immediately.

28220968• 5 years ago

If someone asked you in-person "when did Neil Armstrong set foot on Mars?", would you just say "July 20, 1969"? Or would you say "nobody has been to Mars, but if you're talking about the moon..."

Google's response here only makes sense if Google said "Did you mean: When did Neil Armstrong land on the moon?"

jcranmer2• 5 years ago

There's a few issues here.

The first is that Google has specifically chosen to call out an answer in some kind. If the query is reasonably framed as a question, there is a clear indication in the UI that the response is meant to be an answer to that question.

Now it's definitely the case that a lot of questions have some amount of semantic ambiguity that a listener would have to resolve. For example, a question about a "president" can reasonably be inferred to mean specifically a "US president" of some kind, at least if the query is from the US and is in English.

And sometimes people can ask questions where there's a confused detail. And responding with the question they probably meant to ask is not unreasonable.

However--and this is a big however--it is incumbent to emphasize that the answer is for a different question than the one that was literally asked. You see this when you do searches of misspelled terms: "did you mean this one instead?" Because occasionally, no, you did mean the term that has much fewer results.

And this kind of emphasize-the-answer can have poor results sometimes. Ask Google which president became supreme dictator. The answer makes it clear why it thinks that, but... that's a really different question from the one that was asked.

outworlder2• 5 years ago

This is actually a really good example of the limitation of current ML/NLP approaches, that there isn't really any level of 'comprehension' at all.

That happens even with humans, so I'm not sure that follows. "Oh, sorry, you meant Mars, I heard Moon"

wombatmobile• 5 years ago

That happens even with humans, so I'm not sure that follows.

When it happens with humans, it's also an example of non-comprehension, possibly for a different reason.

We don't see things as they are, we see them as we are.

Actually, perhaps that's the same reason.

a_square_peg2• 5 years ago

I mean that there is no semantic comprehension or knowledge of 'Neil Armstrong' (as a specific person), 'land' (as an action), and 'Mars' (as a place). It's not a mishearing because computers don't mishear ASCII-inputs like we might do with auditory signal.

I get the same result when I Google 'when did Lance Armstrong land on the moon', 'when did Buzz Lightyear land on the moon', or 'when did Lance Armstrong land on Mars.'

gameswithgo• 5 years ago

It would be more charitable to say that current ML/NLP approaches have orders of magnitude less understanding than humans. The ML/NLP process just jumps straight to the correlated answer, which a human would do, but then use higher order reasoning - "wait mars? that wasn't where he set foot".

This to me is similar to current autonomous driving limitations. The cars can only respond to situations they have seen before. Any novel elements lead to failure, where a human can fall back on higher order reasoning: "That is a stoplight yes, but it is on the back of a truck, ignore it" (real example)

patrakov• 5 years ago

In a school where I studied, something similar was used as a trick question during a history exam. "Which language Vladimir Lenin used to write correspondence addressed to Karl Marx?" or something like that. Nearly half of the class failed on this. To those unaware: Lenin discovered Marx's book, Capital, in 1887, while Marx died in 1883, so there could not be any correspondence.

mikewarot2• 5 years ago

More appropriately, you could ask which language did Karl Marx use to write fan mail to Abraham Lincoln? ;-)

boublepop• 5 years ago

If you ask the average American “When did Niel Gaiman set foot on the moon?” Most would answer that they don’t know exactly but think it was in the 60’s.

This is not a limitation of AI, it’s exactly what you want it to do. It’s reading into the context of the question and finding it more likely that you made a mistake in your question than seriously want an answer for a constructed nonsensical question that has no frame of reference or context in our common knowledge pool.

If you want exact logical answers deduced from base prepositions you don’t want ML models or “AI” your looking at formal logic and deduction.

rasz2• 5 years ago

I dont think comparing yourself to an average idiot on the street is what google should strive for.

The way it used to work you had one intelligent entity forming a query and a machine performing said query. Somewhere along the line Google got this idea returning no result was a bad outcome and started overriding queries. Its like you go to the hardware store and ask for nails, but shopkeeper starts telling you all about nail saloons in the vicinity because they ran out of finishing nails - or worse, goes nail saloon -> spa -> finishing -> ending -> happy ending.

We went from intelligent entity forming a query and a machine performing said query to idiot computer treating you line an idiot.

dane-pgp2• 5 years ago

Most would answer that they don’t know exactly but think it was in the 60’s.

I think this is an important point. Humans and Google have a similar bias (for different reasons) that they want to be helpful or seem knowledgeable when asked a question, so won't say "I am confused by that question" and will rather give an answer to a question which is similar to a question that they can understand, in the hopes that it is approximately right.

In the case of asking when Neil Armstrong landed on Mars, guessing "I think it was in the 60's" is accurate to within 10 years but off by 56 million km. For a good example of what average people think about the solar system, though, consider the question "Is the moon really a planet?" that was once part of an impromptu debate on TV:

https://www.cnet.com/news/qvc-stars-confused-about-whether-t...

m1117• 5 years ago

Maybe it's a conspiracy. They say apollo went to the moon, but in reality it was mars.

AnimalMuppet• 5 years ago

And the big secret that they're hiding is that they were able to journey to Mars in three days. To this day they say that such a time is impossible, that it would take months.

/s (in case anyone couldn't tell...)

technothrasher• 5 years ago

Three days? Pfft, all I had to do was go to sleep in a cave in Arizona...

ur-whale• 5 years ago

So Google does not understand the question ... not exactly a surprise.

TchoBeer3• 5 years ago

Or it assumes that you were asking the more sensible question of when he stepped foot on Mars.

titzer• 5 years ago

Please don't post links to (live) Google searches. It will absolutely give different results depending on the entire state of every computation everywhere, simultaneously. In two weeks, this will return who knows what.

Please take a screenshot for posterity.

BitwiseFool• 5 years ago

One of the things that bothers me about modern Google search results is that I can't be sure the other person I'm telling to google something will even see the same results as me. Results change over time, sure, but google can't help but try and tailor results.

MereInterest• 5 years ago

I'm similarly bothered by rolling updates of any software. If something changes, there's no version number to see what documentation is applicable. There's no way to test on a different computer to see if it's my local environment. There's no way to roll back to a previous version to see if one of them broke.

I understand the advantages of A/B testing from a development side, but it makes for a miserable user experience.

Abishek_Muthian2• 5 years ago

Not even different person, Even if you search a query multiple times in a row it changes results drastically. I got a phished website of my Govt's tax website as a top result due to this[1].

W.r.t OP, I've been contemplating a search engine wall of shame[2] for a while now for recording poor search results and perhaps provide actionable feedback to the search engines.

[1] https://twitter.com/Abishek_Muthian/status/14097614168657469...

[2] https://needgap.com/problems/207-search-engine-wall-of-shame...

geofft• 5 years ago

You put too much faith in Google. https://www.google.com/search?q=when+was+running+invented has been telling you the story of the 18th-century genius Thomas Running for years.

spoonjim• 5 years ago

LOL. I'm imagining cavemen walking briskly away from elephants and bears Olympics-style because running simply hadn't been invented yet.

cyounkins• 5 years ago

“Running was invented in 1784 by Thomas Running when he tried to walk twice the same time”.

Seanambers• 5 years ago

That's hilarious, and also symbolizes how bad google (have gotten?)is.

easton2• 5 years ago

For said future posterity: https://i.imgur.com/EhPpnYX.png

belter4• 5 years ago

Thanks easton

arthurcolle• 5 years ago

How did you dark mode-ify your Google?

thewakalix• 5 years ago

https://archive.vn/aOXKA

lstamour• 5 years ago

For future visitors after Google fixes the bug: at this time, Google gave the same response that you would expect if the query mentioned Earth’s moon instead of Mars. This is also probably true if you ask Google the same question as an “Okay, Google” search. It’s a big info box that says when the moon landing was, shows a photo of an astronaut on the moon, and so on.

Edit: corrected, thanks :)

morganvachon• 5 years ago

...if the query mentioned Earth instead of Mars

Did you mean to say "the Moon instead of Mars"?

mynameisvlad• 5 years ago

Pretty sure you meant the moon. I would guess he first stepped on Earth closer to his birth date (although I'm guessing his first steps were a few months after ;))

akiselev• 5 years ago

I don't think it matters with knowledge cards - the system is so fuzzy that it'll throw up that answer to almost anything.

Apparently Louis Armstrong, the legendary jazz musician, was there with Neil Armstrong on the Moon in July 1969 as the oldest astronaut in history, a few years before his death at the age of 69. [1]

[1] https://imgur.com/a/0JyDB70

belter4• 5 years ago

[0] when did google shutdown?

[0] https://www.google.com/search?q=when+did+google+shutdown%3F&...

FridayoLeary• 5 years ago

It will return this page.

notRobot• 5 years ago

Screenshot for posterity: https://imgur.com/a/Jn0ARXq

okareaman• 5 years ago

when did Napoleon Bonaparte invade japan?

bellyfullofbac• 5 years ago

I tried the original query with Lance Armstrong and Louis Armstrong. Google still highlights the 1969 date: https://i.imgur.com/R3A2E6Z.png

Asking my phone via voice returns a sentence that starts with "Armstrong..." and talks about Neil's moon walk.

This is... a bit aggravating.

Man, referring to the story the guy whose photo Google attached to the text talking about a serial killer with his name, what if Google returns a question about him with a sentence about the serial killer?

E.g. something like "What is Barack Osama famous for?" "Osama is well-known as a terrorist". At least most of the world knows this isn't true, again what if it's Joe Neighbor, who shares a name with Joe Child Molester...

salted-fry• 5 years ago

In a similar vein, some time ago I tried to search for how many unicode code points there are with "How big is Unicode?" (https://www.google.com/search?q=how+big+is+unicode)

Google helpfully responds "16 bits", which is pulled from the History section of Wikipedia and hasn't been accurate in something like twenty-five years.

Edit: Should have listened to people saying to screenshot your queries. Google still quotes the paragraph in question, and bolds "16 bits", but no longer puts it in a big bold heading like it's the single answer to your question.

Double Edit: except in chrome, where I do still get the old page. Here's a screenshot for posterity, after Google somehow fixes this: https://i.imgur.com/7Ng6DyK.png

mikewarot2• 5 years ago

UTF32 is the way to go for internal storage, until you pack it back down to UTF-8 to store externally.

tim333• 5 years ago

I get a different result "Unicode uses between 8 and 32 bits per character" https://imgur.com/a/hxmrMz3

rasz2• 5 years ago

Its like "rick and morty season 5" returns

    Number of episodes: 8
    No. of episodes: 6

garbage in garbage out

silisili• 5 years ago

I think I saw it here, but perhaps elsewhere...

If you want a weird ride, ask Google 'how many raccoons fit.' A bit NSFW.

Imnimo• 5 years ago

I'd imagine Google's position is that no one ever searches for this in earnest, and so it doesn't really matter what the answer given is. They want to maximize what percentage of actual searches give a correct answer, rather than what percentage of possible questions do.

On the other hand, it's a dangerous game. You never know when current events might make a previously never-searched question with a wrong answer very popular.

TchoBeer3• 5 years ago

Yes, I'd imagine the number of people searching for the year Neil Armstrong went to Mars because they for some reason are extremely deluded about history is much, much smaller than the number of people who type "mars" when they meant "moon"

eitland2• 5 years ago

Problem is Google does this when I search for a very specific, weird error message too, in which case I actually mean that specific weird error message.

I've lost a few hours to Google trying to outsmart me and it is maddening: not only do I waste hours but it is not given that I find the magic incantation that gives me the answer.

DuckDuckGo.com is getting better lately but is not stable. Lately I've also started using metager.org which has seemed promising so far.

carlsborg• 5 years ago

Is this a feature or a bug?

sschueller• 5 years ago

Ask Google how tall Elon Musk is. You will get 188 cm and conversion of 5 foot 10 inches which is way off. Two websites in the results that feed Google have the conversion wrong. Bad data is all over the place and copied over and over even simple incorrect conversions.

1970-01-01• 5 years ago

When did Halley's Comet collide with the moon?

Awesome. I must of been busy that day, I would of remembered that.

edit: its been fixed. here it was: https://ibb.co/myxWK3C

outworlder2• 5 years ago

I must of been busy that day, I would of remembered that.

As a non-native speaker, I'm having trouble understanding the sentence, specially the "I must of", "I would of" constructions. Is this some style choice I don't know about?

gabrielsroka• 5 years ago

I must have been busy... I would have remembered that.

I must've...

Have sounds like of, so it's often written incorrectly.

MiddleEndian2• 5 years ago

Still shows up as 2013 for me, although directly under it explains that it's a "what if" situation. https://i.ibb.co/HDh1SG4/image.png

travoc• 5 years ago

AI-driven content will become the next scourge of humanity, just like engagement-driven social media feeds and internet ad tracking. Thanks, big tech.

sorokod3• 5 years ago

"When did Artemis 11 landed"

July 24

lkbm• 5 years ago

I think it's fixed now, but I came across this one a few months ago: https://twitter.com/lkbm/status/1330598466989543425

("First woman in space" result snippet is about Sally Ride, the first American in space. First result, below that is the Wikipedia entry for Valentina Tereshkova, the first woman in space.)

MiddleEndian2• 5 years ago

https://old.reddit.com/r/YellowSonic/comments/p6d4f7/thanks_...

Saw this yesterday, my Google search results (regular and image) are basically the same. A bunch of the top results for "desk ornament" are nazi memorabilia for some reason:

https://i.ibb.co/2Y02FZv/image.png

wila• 5 years ago

Not just google, ddg has a similar list.

How did this happen?

polytely2• 5 years ago

I really wish someone would come along and kick googles ass on search, it's embarrassingly bad. It's wild how I used to look up to the FAANG folks, but it has become a real emperor has no clothes situation.

autokad• 5 years ago

this one is a bit problematic, because people assume those 0 click search results are True / gold standard. like when I do 37 c to f and I see it display the results, I never double check them, I just assume they are right.

polytely2• 5 years ago

Yeah the fact that *this* is what the mighty Google just let loose upon the world shows that they don't really care any more about their users, their products and their supposed mission of organizing the worlds' information.

The fix is quite easy too, just have an AI driven system with human supervision, I think at this point it is well known that a human + AI will basically always trump an AI and it's not like google doesn't have the resources to pull this off. You could build a database of verified facts that are often googled, have the AI gather data, have humans do a sanity check.

It's probably just some scheme to get someone promoted inside Google, the infobox is a side-effect of someone's career plan, it doesn't really matter if it actually works as long as it hits some key metric that gets used in an evaluation somewhere, who cares about propagating misinformation and falsehoods google is still making money. It makes me depressed.

ChrisArchitect• 5 years ago

this is a perfectly good response because chances are that's what the user meant/was looking for (also since the scenario of Armstrong on Mars doesn't exist)

firebaze• 5 years ago

I wonder if SHRDLU* would understand such queries better :)

* https://en.wikipedia.org/wiki/SHRDLU

max_• 5 years ago

Wow. I just tried "When was Obama killed" and it returned me a date.

[0] https://www.google.com/search?q=when+was+obama+killed

belter4• 5 years ago

Obama needs to shave and maybe see a doctor about that Vitiligo...

freediver• 5 years ago

As someone who is building such instant answer system for our own web search engine [1] the level of failure here tells me that this is:

- Either a basic, distilled BERT based model (optimized for latency and scale) or

- More likely, still a purely heuristics driven answers system like the one Google has been using for last 10+ years.

The current NLP models are able to quite successfully answer these questions with proper context. In this case ignoring Lance or Mars in question looks more like old-school keyword based heuristics and there is no way it can get this right.

Google is not only allowing mistakes in this, what is considered a fairly difficult problem to solve, but also questions that directly query their own knowledge graph, for example this one querying for a CEO of a well known public company where it returns the wrong answer [2]

This only shows that 'emperor has no clothes' [3] and that there is still a lot of room for innovation left, specially on the 'organizing the world's information' front.

[1] https://kagi.com (currently in private beta)

[2] https://ibb.co/qkXpdFB

[3] https://www.quora.com/Is-Google-really-in-a-decline (almost 2M views for the top answer indicate that 'Is Google in decline" is a fairly popular question among presumably Google users)

ttonkytonk• 5 years ago

This reminds me of how recently Google did not make it clear that there is no longer a tax penalty for not having health insurance (except in five states and D.C. I think).

When did Neil Armstrong set foot on Mars?

Related Stories