Kogan.com mysteriously disappears from Microsoft Search Engines

http://www.kogan.com/au/blog/Kogan-dissapears-from-microsoft-engines-bing-yahoo/

by _Mark • 14 years ago

105 48 14 years ago

DeepDuh• 14 years ago

Maybe their crawler identifies as IE7?

nekgrim2• 14 years ago

Kogan's website isn't blocked for IE7, so I could not see how it can be a problem if the crawler identifies itself as IE7.

raverbashing• 14 years ago

It is a problem if the site presents different content (that is, for example, the light box announcing the tax)

This may throw the search engines away, because you're "de-SEOing" your content

schiffern• 14 years ago

Nope.

http://www.bing.com/community/site_blogs/b/webmaster/archive...

Killobyte• 14 years ago

"We never waged war against Microsoft over IE7, we simply wanted people to upgrade their web browsers..."

Maybe so, but the splash says "Use a better browser" and lists Chrome, Firefox, Safari, and Opera. It would be ridiculous for Microsoft to remove the page from their search results because of this, but at just a glance, and maybe to the untrained user, it could look like the are waging war on IE as a whole, and not just trying to get users to "simply upgrade their browsers."

pyre• 14 years ago

IIRC, XP will only ever go as high as IE7 or IE8. Vista will only go as high as IE9. Anyone hanging around on those operating systems, will need to look outside of Microsoft for a more up-to-date browser.

raghavsethi• 14 years ago

That true. They could've simply linked to the IE9 download page.

But I think MS's algorithm would have de-ranked the site because any IR algorithm considers the content of the page too - if the web page served to IE users is about redirecting to browser download pages, their crawler (which may identify itself as IE) might not consider it to be the best result about 'Kogan'.

Jacobi2• 14 years ago

When I perform a search on bing the first result is the kogan website ...

EDIT: It was the kogan.co.uk and not kogan.com since my country/region = UK

Jabbles• 14 years ago

Neither "kogan" nor "kogan.com" return the website for me.

Jacobi2• 14 years ago

This is a screen shot of my search results http://i.imgur.com/If1E2.jpg

DanBC• 14 years ago

I'm in the UK. I get the UK site as the first hit. I don't get the .com.au sites, even if I search for [kogan.com.au]

I've seen a few threads on HN where someone claims that Google has removed them from the search results (often after a Google update) and it turns out that the site had a weird set of errors. (mix ups about http and https, and about including or not the www.)

mtgx• 14 years ago

I think Microsoft themselves admitted that they do a great deal of manual editing for their search engine rankings.

rplnt• 14 years ago

Where? By quick search I did not find anything.

sreyaNotfilc2• 14 years ago

I think mtgx is referring to where Google claimed that Microsoft stole their search results...

http://googleblog.blogspot.com/2011/02/microsofts-bing-uses-...

citricsquid• 14 years ago

Microsoft themselves want their users to upgrade to latest browsers (see: http://www.ie6countdown.com/) so the idea that they would take issue with someone else doing this (so much so that they'd remove them from bing) seems a bit.. unlikely.

Maxious• 14 years ago

For those who are not familiar with Mr. Kogan: http://en.wikipedia.org/wiki/Kogan_Technologies#Controversie...

As another commenter suggested, I would not put it past him that he blocked bing somehow just for a publicity stunt. The wikipedia article hasn't even caught up with last week's stunt (or there are just so many that it cannot list them all): http://www.gizmodo.com.au/2012/07/kogan-launches-campaign-to...

jawr• 14 years ago

I like these guys, particularly for the 'Website glitch', I can't imagine many retailers would take a similar stance.

Seems to me that these guys are great marketers.

AshleysBrain• 14 years ago

I think this is just some random technical glitch that has de-ranked them for some reason at an awkward time, along the lines of Google accidentally marking a legit website as spammy, sending it way down the results. Can it really be proved that this was done directly in response to the IE7 tax? What would the point be anyway? Maybe the storm of incoming links about the IE7 news accidentally got identified as linkfarming somehow?

glesica• 14 years ago

Interesting that it comes up in DDG, which I thought was partially backed by Bing. They augment the results quite a bit, though, so not terribly surprising.

Also, I made myself sad when I typed "kogan.com" into the address bar to test DDG and was shocked when the actual web site came up. Didn't think that one through...

gary4gar• 14 years ago

If this is real, then Microsoft would is going to get into a long legal battle. I hope this is some technical glitch & not some random angry guy at Microsoft

freehunter• 14 years ago

There's only been one case I'm aware of where a site was penalized in search rankings and filed a lawsuit. That site lost. The only complaint that the judge seemed to find valid (although ultimately dismissed) was anti-trust allegations. Microsoft wouldn't be in that position in search.

In short, there's no legal precedence for filing damages if you're dropped by a search engine. It would be hard to imagine what, if anything, Kogan would argue successfully.

http://blog.ericgoldman.org/archives/2007/03/kinderstart_v_g...

magoon• 14 years ago

I use Bing, so this is disappointing - but we can only assume it is a mistake, and the most interesting part of this story will be how quickly Microsoft reverts & responds.

nhebb• 14 years ago

Funny, the ad I see displayed with the search results is:

  Upgrade from IE6 Today
  Microsoft.com/IE9 Say Goodbye to Outdated Browsers.
  Download the Latest IE Browser Now!

gouranga• 14 years ago

I reckon this is because the ranking algorithm deranked them when their click through from bing rate declined after the sudden publicity storm. Their #visitors delta probably went very negative after everyone stopped clicking links from various news sites.

They probably shot themselves.

jonursenbach• 14 years ago

Doubtful. Even searching "site:kogan.com" pulls up zero results while "site:kogan.co.uk" pulls up 1.9k results.

Kogan.com is completely gone from their resultset.

damian2000• 14 years ago

Ruslan Kogan must be rubbing his hands with glee at this development - more pricele$$ free publicity for his site.

chrislomax2• 14 years ago

Anyone else thinking they added a robots.txt entry for bing to disallow, waited to get de-listed then removed the offending robots.txt entry, posted this news article knowing bing would pick them back up in a couple of days?

jbigelow76• 14 years ago

If that's what they did (no telling really) that would be pretty clever PR wise.

rmc• 14 years ago

Clever… and dishonest.

michaelt2• 14 years ago

The HTTP last modified header for www.kogan.com/robots.txt has Last-Modified: Wed, 06 Jun 2012 06:14:35 GMT

It's possible they did this and they were cunning enough to turn back the last-modified clock.

reinhardt• 14 years ago

How did you get that? Curl says May 4 instead:

  $ curl -I http://www.kogan.com/robots.txt
  HTTP/1.1 200 OK
  Server: nginx/1.0.14
  Date: Mon, 09 Jul 2012 18:48:34 GMT
  Content-Type: application/octet-stream
  Content-Length: 79
  Last-Modified: Fri, 04 May 2012 18:58:02 GMT
  Connection: keep-alive
  Accept-Ranges: bytes

blauwbilgorgel3• 14 years ago

They serve their robots.txt as a "application/octet-stream" not "text/plain". Might be confusing some search engines?

Also according to Internet Archive their robots.txt has changed in the last months.

This was a version that was archived:

http://web.archive.org/web/20080104044407/http://www.kogan.c...

At a glance, they are whitelisting some bots, including Googlebot, but not including Bingbot, and disallowing the rest.

  User-agent: Googlebot
  Disallow: 

  User-agent: *
  Disallow: /

Another marketing stunt?

UnoriginalGuy2• 14 years ago

Your link is pointing at 2008. That is four years ago. Hardly proof of anything.

You are right about the content type however.

blauwbilgorgel3• 14 years ago

Correct. It doesn't proof much, but that they previously had a buggy robots.txt.

Usually these posts are accompanied by a declining traffic graph for Bing referrers. That would give us more proof and a timeframe to work with.

The post gives two options for their absence in Bing results: a fault at Microsoft or as a punishment. With their history I think a fault on their own part is a more likely option. I can't rule that out.

brudgers• 14 years ago

Just a thought, but I seem to recall that Google was giving higher rankings to sites that advocated upgrading to Chrome, and created a bit of embarrassment.

Given that Kogan.com advocates upgrading and all the linking to their website which occurred recently, perhaps it seems possible that a Microsoft algorithm designed to avoid similar SEO manipulation was triggered.

In other words, the way in which the Kogan.com story unfolded was sufficiently similar to the PR move by Google, that it triggered a Microsoft safeguard.

http://www.pcworld.com/article/247257/google_disciplines_its...

chrislomax2• 14 years ago

They did this to themselves as it's against their guidelines to pay for page ranked incoming links. It was simply because in the end they were paying for the link to their site

UnoriginalGuy2• 14 years ago

Their robots.txt file is being delivered as the wrong content type for me. It is being served as a "application/octet-stream" instead of a "text/html"

I have no idea what consequences this might have since I've never seen another site which does it. Some quick Googling turned up nothing.

Maybe Bing choked on the wrong content type and decided to blackball their entire domain in order to avoid indexing the unindexable?

X-Istence• 14 years ago

Why would you want to serve up a robot.txt as "text/html" when in reality it is "text/plain"?

ot• 14 years ago

"we noticed that the www.kogan.com website stopped appearing in organic search results on Microsoft backed search engines"

They don't say explicitly, let alone prove, that the result ever appeared on Bing, while "stopped" would imply that.

If the comments about the wrong robots.txt got it right, it is well possible that the website was never indexed by Bing, and they just noticed it (or decided to talk about it) now.

Isofarro• 14 years ago

Looks like Kogan only serves the UK and Australian market, there's no support from the US. Perhaps this is a ranking correction to delegating Kogan pages to UK or Oz specific searches rather than global or US.

sreyaNotfilc2• 14 years ago

This "tax" is elitist internet bullshit.

I'm all for using any browser that you'd want, but why go through elaborate lengths to put Microsoft down? Especially on a version of IE that's 3 iterations in the past.

I understand that IE7 may not be as great as the other browsers. I'm fine with being annoyed by that. But what about the other versions of IE? Why isn't there a link for IE9 or IE10?

whatusername• 14 years ago

This "tax" is internet marketing linkbait. Pure and Simple.

Kogan.com mysteriously disappears from Microsoft Search Engines

Related Stories