Compare & vote for search results of Bing, Google and Yahoo in a blind test

http://blindsearch.fejus.com/

by vaksel • 17 years ago

123 41 17 years ago

blindsearch.fejus.com

viraptor• 17 years ago

I'm actually surprised - my guess would be something like <10% for yahoo and almost equal results for google and bing.

On the other hand the test can't take one important feature into account - profiled results. After 2-3 searches for anything programming related on someone else's browser I feel there's something wrong with the results... yep - I'm not logged in. Even if it doesn't make a huge difference, it is noticeable. I'd really like to check the stats while being logged in into my google account.

carbon84• 17 years ago

1. This was created by a Microsoft employee (a "Microsoft Developer Evangelist"). Maybe this doesn't matter to you, but considering the $100m marketing budget and the widespread use of astroturfing, it seems a little dishonest to me (certainly treading in a grey area) that there isn't full disclosure. YMMV

2. As someone who cares deeply about research methods, I'd argue that there is little to no value in superficial, subjective and contextless comparisons like this.

Edit: He has now added a statement that he works for microsoft. It was not there before I posted this comment

vaksel2• 17 years ago

Well if you question the methods, you can always open bing/google/yahoo and check to make sure the column labels are correct

carbon84• 17 years ago

I didn't say, hint or imply that it was listing inaccurate results. I stated that it's superficial, subjective and devoid of context. Frankly, it means nothing.

For example, someone searches for "bongo". The forth results are a radio station, a page about the animal and an email/calendaring system. Given those results, someone votes for one. Why? We don't know. They might barely now. One thing we do know: it was a one-off search anyway, not something that was part of someone's normal workflow.

brendano• 17 years ago

<i>2. As someone who cares deeply about research methods, I'd argue that there is little to no value in superficial, subjective and contextless comparisons like this.</i>

I care deeply about research methods too, but I don't understand why this is bad. If the goal is to know which search engine is best -- a subjective question in the first place -- it's gathering judgments from people on exactly that topic.

Of course, you can't take the statistics collected overly seriously, since it's hard to know how seriously different people are taking it and such. But it's probably more true than not.

TweedHeads• 17 years ago

I tried camino, jetpack, svg, xul and never voted for bing.

As long as MS tries to move down relevant links to competing technology their search engine is useless.

screwperman• 17 years ago

Seems like I've finally found a way to Bing for "sex" in India.

froo2• 17 years ago

Seems like I've finally found a way to Bing for "sex" in India.

Actually I found out you can bypass the filter by adding in a generic stop word at the end which the search engine should ignore anyway (I use the word "the") and it will search for whatever phrase you want just fine.

Discovered this about 8 hours ago - http://news.ycombinator.com/item?id=645812

amichail• 17 years ago

The methodology for this is flawed because people will tend to search for things they have searched for before on Google and then pick Google as the best because they find that ranking more familiar.

socratees3• 17 years ago

I don't think its flawed. How can you say which results belong to Google or any other search engine?

bemmu• 17 years ago

For example I searched for "japan", and expected to find "japan-guide.com". Now I realize the only reason I even know about that site is because it was in my Google results before, so now I grew to consider it the "correct" answer.

pg• 17 years ago

I tried 5 searches about subjects I knew well. Results: Google 4, Yahoo 1, Bing 0. But in every case I had to look closely at the results to decide which I preferred. None of them returned obviously bad results for any of the queries.

okeumeni• 17 years ago

You should share search text so all can appreciate your results.

prakash• 17 years ago

Do you usually search for information on subjects you know well or subjects you don't know well?

One needs to use any of these search engines over a period of time before making a switch, mostly because the improvements are incremental in Bing.

diN0bot• 17 years ago

I searched for Patti Blogovich (intentionally misspelled). All three columns contained what appeared to be relevant results, but after reading the returned articles, only two articles were outstanding (on depth and "objectivity"). One of these was halfway down the column for what turned out to be Google. The other was the top result for Yahoo.

Then I searched for "wriggly wrolly," which I sometimes call my dog. I ended up clicking for Yahoo again, which had all 4 results.

Sure, only two tries, but I was really surprised with how rewarding this experience was. I wish I could use Yahoo but have it look like Google. I'm just so hooked on Google's branding and image, rather than my 90's-based judgments of Yahoo...maybe I can be stronger of that and make the switch...maybe.

socratees3• 17 years ago

This application is weird in cases where spell correction comes in to place. Just search for mike tieson, birack obam, pal graham hacker news and you can see the difference between google results & what's shown on the blindsearch application. There's a whole lot of difference.

dsil• 17 years ago

Cool site, but not a real comparison. I know google does a lot of personalization, based on being logged in or your cookie, but it can't do that on this. I'm assuming the other engines do something along those lines as well.

sachinag2• 17 years ago

For someone who's looking to SEO their site, this tool is exceptionally useful just for that reason.

sachinag2• 17 years ago

This is about as useful as Yuil was - i.e. very. Sure, it turns out to be a little self-serving for the person's employer, but you have to acknowledge the brilliance of the effort and the fact that the results themselves appear to be on the up-and-up.

drp• 17 years ago

This is great, but I felt like my mental comparison weighted each individual result more equally than I normally would in a routine search. Another interesting comparison would be to show only the top result, top 3, etc. from each search engine.

dangoldin• 17 years ago

Looking at the source code you can find which site is used for which column when "engines" is defined: var engines=new Array("YHO","MSF","GOO");

Voting will just reveal that information.

Although if you wanted to game the site you could already do it in different ways.

fizx• 17 years ago

My score: Google 3, Yahoo!! 3, Bing 1.

The test is to a large degree unrealistic. Do a search for "sf restaurants". Google and Yahoo!! (in a recent upgrade!) both have extremely useful OneBox/Shortcut results that aren't captured in the comparision.

froo2• 17 years ago

I did 10 searches about subjects I was passionate about. Results

  Google: 5
  Bing: 5
  Yahoo: 0

Having read what other people were writing in this thread I thought I would also try and rank the results in order, which would be my first choice, which would be my second choice - so for second choice results, here they are.

  Google: 3
  Bing: 5
  Yahoo: 2

So while the Google/Bing split for first place results were interesting, the second place results were more interesting (to me at least).

I'm going to keep throwing in results later today as I think up more topics to increase the sample size a bit to see which is a bit better, as I think it will trend towards a more even split or even a Google lead, but definitely promising results for Bing.

EDIT - Update, I just checked the results again, seems like it was just hacked in Yahoo's favour.

Around an hour ago (when I first published this comment) I noticed it was about 26.5k votes cast with the split being Google 39%, Bing 31%, Yahoo 30%.

Now having a look the split is Google 31%, Bing 25% and Yahoo 44% with 40k votes cast... I suspect shenanigans.

EDIT #2 - 2 minutes later, split is Google 30%, Bing 24%, Yahoo 46% with 41.5k votes cast, so looks like the fixing is in process. Wonder which Yahoo engineer is doing this...

EDIT #3 - 6 minutes after edit 2, Total is 45k, splits are Google 29%, Bing 23%, Yahoo 49%... Any insight that could be gleamed from this rough poll is now officially tainted, nothing more to see here.

grandalf2• 17 years ago

which do you choose?

http://blindsearch.fejus.com/?q=sex+tape

grandalf2• 17 years ago

uh it's interesting b/c only one search engine ranks wikipedia first!

jsz0• 17 years ago

In my personal tests the results are almost identical but the order of results may vary by a few spots. The most relevant searches are appearing in the first page so the order isn't all that important IMO. If anything this proves to me that accuracy is no longer the big deciding factor in search for most people. It's all about the platform -- mail, IM, maps, blogs, photos, etc. This explains why Google has stayed strong since they've made the biggest investment in a complete, cohesive, platform.

davidalln• 17 years ago

Search itself is simply too hard to judge. Since many of the results in the top 10 (at least in my test searches) were the same but perhaps jumbled a bit, I personally think it doesn't matter that much.

Day to day I use search for two things: very general information and looking up code errors. Code error searches will pull up the same mailing lists and forums no matter which search engine you use, and a general information search will either pull up an official website or a Wikipedia page on any search engine. 9 times out of 10 that's sufficient.

There's no superficial reason to why I use Google over Bing or Yahoo!, just simply that I've always used it and I'm always satisfied with it. Yes, I also like its interface the most and non-search related Google applications are generally amazing, but I wouldn't be any less productive if suddenly I switched to Bing for all my search.

(Although I might simply just be biased towards Google... they did just give me a free phone!)

I think the 40/30/30 results that are currently posted on there show that. For the most part it's relatively even, and honestly Google is probably winning just because some die hards are bent on making sure their favorite wins.

IsaacL• 17 years ago

Tried three tests: my own name, "Clojure tutorials" (something I've been looking for recently), and ""ORG directives in assembly code" (something I had trouble finding recently). Scores - Google 1, Yahoo 1, Bing 1. Not a very large sample space, to be sure, but still it was very hard to tell the difference, and by the stats, a lot of other people have had similar reactions (with a slight lead for Google).

Maybe this means we should just stick with the company we like the best? Google it is then ;)

mortenoffline• 17 years ago

I've been using the WebMynd plugin since they posted here a while ago. Been searching for normal stuff + django development. Yet too see a better top 3 result from Bing. The ranking is a lot worse, might be that they need a bit of time for tuning. To me it seems to be something wrong with tokenization / word distance weighting.

socratees3• 17 years ago

He also needs to have a way to mark the worst result out of 3.

anirudh• 17 years ago

Yahoo seems to be rising incredibly fast. Its greater than 50% now.

randomwalker• 17 years ago

As others have noted, the site is being gamed. Someone is stuffing votes for Yahoo. Rather unfortunate, because otherwise it was interesting and useful.

jsrfded• 17 years ago

The vote counter is going up like crazy now, and Yahoo is now trumping both Google and Bing. Yahoo was in 3rd place earlier today when I looked. Seems like someone is hitting it with a robot to goose Yahoo's numbers. So much for the data.

jpwagner• 17 years ago

I use google almost exclusively.

I only used Bing as a test on the day of its launch.

That said, when I chose based on the results: Bing 70%, Google 30%, Yahoo 0%.

Wow...

Compare & vote for search results of Bing, Google and Yahoo in a blind test

Related Stories