Belgian newspapers: Google blocking us on searches

http://www.google.com/hostednews/ap/article/ALeqM5jj9ad340cxT7i1ADyPk_ksWRtLgQ?docId=79891bacb5a84c068862fba59868b1d6

by robtoo • 15 years ago

39 50 15 years ago

google.com

itcmcgrath• 15 years ago

The only comment that comes to mind: Don't bite the hand that feeds you.

The newspapers really didn't think that one through, did they...

shrikant• 15 years ago

I suppose to extend that analogy: the newspapers didn't believe they were biting the hand that was feeding. They saw it more as cutting off the other hand that pushed their heads towards the food.

Google is saying both hands are equivalent, but the newspapers are saying they want one hand to feed them, and the other to keep away.

I can buy that, actually. Sure, I don't agree with the initial ruling, but IMHO de-indexing the newspapers from the search engine was unnecessarily retaliatory. I can understand why Google did it, but having made their point, I think they should gracefully back down claiming ignorance. This is a clear misuse of their 'monopoly' on web search for benefit in other areas.

EDIT: Oh good, my Google fanboi-ness is not in vain. The court ruling actually ordered that links be removed from "Google Web [sic]" as well. Oh well, the joke's on the newspapers then, as it appears they asked for it! (see roel_v's comment: http://news.ycombinator.com/item?id=2776384)

yock• 15 years ago

Unnecessarily retalitory? I wouldn't be so sure. Don't forget that Google search results include selected content from indexed websites. The index itself actually contains all content, which Google may have believed to be a litigation risk given the vagueness of the removal request.

turtle4• 15 years ago

Google supplies the tools for the newspaper to have marked explicitly, using technology, the pages they want included/excluded from search, with no ambiguity. Instead, they chose to use vague, court language, with threat of a fine per link, to force google's hand. Personally, I don't know why google -wouldn't- drop them, permanently.

There is no monopoly on web search. Search works for everyone, specifically because there are established protocols and conventions that allow all these systems to work together. If your company chooses to ignore these conventions, I don't think it is unreasonable that you get ignored.

You don't have a -right- to be included in a web search. The reason google includes you (or any search engine) is because you are making a trade, they get better results which they can monetize via ads, you get more traffic, which you monetize through ads. If you opt out, why shouldn't they? What's in it for them? They aren't running a charity, they are running a business.

muuh-gnu• 15 years ago

> they want one hand to feed them, and the other to keep away.

They dont seem to be in a negotiating position to express "wishes" which then somehow have to be granted in the exact way they want them.

arhughes• 15 years ago

From http://www.copiepresse.be/pdf/Copiepresse%20-%20ruling%20app... Section V:

"Orders Google to remove from Google.be and Google.com sites, more specifically from the "cached" links on "Google Web" and from the "Google News" service, all articles..."

Depending on your interpretation, they were supposed to remove the newspapers from Google News and remove the "cached" links from Google Search results.

melvinmt• 15 years ago

There's a € 25.000 euro fine on every violation of the court order. Google doesn't want to take a chance to leave room for even the smallest amount of ambiguity, so that's why they've removed all links to those newspapers.

ToadyCensored6• 15 years ago

* EDIT: Oh good, my Google fanboi-ness is not in vain. The court ruling actually ordered that links be removed from "Google Web [sic]" as well. Oh well, the joke's on the newspapers then, as it appears they asked for it! (see roel_v's comment: http://news.ycombinator.com/item?id=2776384) *

That's in reference to the Google cache of lead paragraphs and other article excerpts. The lawsuit was not about general links in the search engine. Publishers sued because Google News was displaying lead paragraphs and other copyrighted writing in the aggregator.

If you started your own newspaper that copied all the headlines and lead paragraphs from the New York Times, it wouldn't be long before the New York Times sent you a legal letter.

slater• 15 years ago

Anyone have a copy of the ruling? Might just be the judge(s) misinterpreting what the newspapers were asking for?

roel_v3• 15 years ago

Ruling (in French): http://www.ie-forum.nl/backoffice/uploads/file/IEF%20Hof%20v...

Can't copy and paste from there; relevant passage:

"...orders Google to removing the sites google.be and google.com, more precisely the links (in cache) visible through Google Web and the service Google News (...) of Belgian Francophone and German-language publishers represented by Copiepresse."

So yes, Google is interpreting the ruling correctly, there is no way to interpret it in such a way that the regular google.com shouldn't be blocked. I don't know if the newspapers asked for too much or if the judge didn't understand what he was doing; I didn't read the whole ruling.

masklinn• 15 years ago

> there is no way to interpret it in such a way that the regular google.com shouldn't be blocked. I don't know if the newspapers asked for too much

I think theoretically it would be possible if and only if they specialized Google's crawler and rendering to only render a link (no preview, no cache link) for these sites and these sites only.

Why they'd bother doing that is anyone's guess.

Toady8• 15 years ago

The lawsuit was over the use of lead paragraphs and other article excerpts, specifically in the Google News aggregator, not plain links in the general search engine.

roel_v3• 15 years ago

Update on this story: Google has said they will start indexing the pages for the regular search results again asap.

wcoenen• 15 years ago

Ruling in English: http://www.copiepresse.be/pdf/Copiepresse%20-%20ruling%20app...

In the 2006 injuction, Copiepresse had originally asked the judge to "order [Google] to withdraw the articles, photographs and graphic representations of the Belgian publishers of the French and German-speaking daily press, [it] represents, from all their sites (Google News and "cached" Google or any other name) from the date at which the ruling is served, under penalty of a fine for non-performance of two million euro per day of delay".

And the relevant part of the ruling was thus:

"Orders GOOGLE to withdraw from all its sites (and more specifically from Google News and as far as the Google web search engine is concerned the visible cached links) all the articles, photographs and graphics representations from the authors in respect of whom the intervening plaintiffs can prove that they retain the rights".

In another part of the document, it states that "Google News does not confine itself to placing hyperlinks but reproduces significant sections of the publishers' articles".

If you read the entire ruling, it is quite clear (as far as such a legal document can be clear) that only the copied content on Google News and the cached copies on the search result pages were problematic. Copiepresse or the judge had no issue with hyperlinks.

By taking it further, Google is playing a dangerous power game. It seems to me that this will inevitably lead to a EU anti-trust case.

dexen5• 15 years ago

The newspapers went to court asking for broad relief (unlisting from google results), instead of putting in place proper META tags and /robots.txt to direct what may be, and what may not be indexed and cached. The court handed them just as much of the proverbial rope as they asked for.

Google's web crawlers asked the WWW servers to GET, were served with HTTP 200 OK, found no prohibition in /robots.txt and so followed the standard protocol.

Note Google Web Search and Google News use separate bots (Googlebot and Googlebot-News) to give website owner full control.

reso• 15 years ago

I can't decide whether I find things like this hilarious or disturbing. They could have fixed it with engineering, but they were either so uninformed or incompetent that they tried to fix it with a lawsuit.

pavel_lishin3• 15 years ago

Or maybe one incompetent overrode them all.

Manager: Evans, get in here! Why are our news articles all over The Google?

Evans: Um, sir, they index our -

Manager: I don't want to hear your technical mumbo jumbo! Get legal on this to get it fixed YESTERDAY!

Evans: There's no need, it's literally a 5-minute fix, I could just add an entry to robots.txt while the coffee is brewing...

Manager: What did I just say about mumbo jumbo? Get out of here, and send in Stevens from Legal on your way.

Toady8• 15 years ago

The lawsuit was over the use of lead paragraphs in the Google News aggregator, not the general search engine. Google is intentionally misinterpreting the verdict of the lawsuit in order to punish those who might challenge the company. Remember that Google also defied the court order to publish the result of the lawsuit on its Belgian sites, so it's obviously not afraid to thumb its nose at the courts.

"Nice online newspaper you've got there. Hate to see something happen to it."

Toady8• 15 years ago

The newspapers went to court asking for broad relief (unlisting from google results)

They went to court over Google News copying excerpts from their articles, such as the lead paragraphs, and displaying them in the Google News aggregator. The lawsuit didn't pertain to the general search engine.

Symmetry2• 15 years ago

That might have been the intent of the newspapers, but that wasn't the court order their lawyers got them.

lenni• 15 years ago

Maybe I just don't get it, but why do they want to be included in web search but not in Google News? Their actions seem bizarre.

Both are noindex-able separately, correct?

dexen5• 15 years ago

tl;dr: Yes they are indexed separately, and yes it's up to site owner to decide which content may be indexed for Google News -- via proper META tags and robots.txt.

There is an important difference in UI between Google Web Search and Google News: the Web Search presents a very short snippet of text and link, while Google News presents a bit more of the news content. Effectively, Web Search directs viewers straight to the original website, while News lets them just browse Google News, sending less traffic to the original website. [1]

Which means Google News may provide less ad revenue to original website -- probably the crux of the problem here.

To handle that, Google provides two different bots: one with user agent `Googlebot', the other with `Googlebot-News'. If a website wants its content only in Web Search, and not in News, it's supposed to return proper /robots.txt -- and that does the trick. Way quicker and cheaper than any lawsuit!

http://googlewebmastercentral.blogspot.com/2009/12/new-user-...

----

[1] but hey, less is more! whoever makes it to the original site via Google News has clearly show elevated interest in the news piece, and thus should be worth more to ad providers.

ChuckMcM• 15 years ago

Short answer, one looks like a newspaper.

The longer answer is that the newspapers are struggling financially, Google is not struggling, when snippets of content appear on the Google news page the newspapers believe they are 'improving' the Google brand without being compensated. They sued to have either Google pay them, or remove them from the News site.

What they fail to realize (and Google no doubt knows) is that if you aren't listed in Google you're not going to get nearly as much traffic (hence 'profit'). So Google apparently took a literal interpretation of the order and struck them from the books as it were, complete removal from the index.

So the newspapers have to make the following evaluation, 'what is it worth to us to be in Google's search results?' And is that value more or less than what we think they owe us for having our results on their news page?

Google clearly understands that its value for them to be in the search results and I believe they would say off the record that the value of being in the results is worth more than the ad revenue they get from the occasional story that makes it to the front page of Google news.

Sort of extortion, sort of natural consequences. Humorous though.

bugsy• 15 years ago

OK, it says they filed a lawsuit against Google because they didn't want Google to LINK to their articles without paying a fee for linking. They won the lawsuit in court and so Google removed links to their articles from their search index, as they requested, and now they are mad about it. Why is the story "Google blocking" them. Google's only doing what a court ordered after the papers sued asking for that.

pavel_lishin3• 15 years ago

As I understand it, they wanted Google News to stop linking to their articles; not the regular search results.

dexen5• 15 years ago

From court order, via http://www.chillingeffects.org/notice.cgi?sID=2160

> Order the defendant to withdraw the articles (...) from all their sites (Google News and "cache" Google or any other name (...)

The other articles linked from http://slashdot.org/story/06/09/18/115203/Google-News-Remove... are down now, but I believe the original lawsuit was in that very spirit: remove their articles from all Google's websites. Not just from Google News.

Logical argumentation aside, Google's in no position to second-guess a court order ;-)

Why did Associated Press put such a negative spin to the story? We can only guess...

Toady8• 15 years ago

They sued because Google News was using lead paragraphs from their articles. Google is most definitely block them, because the court's decision didn't pertain to the general search engine. In effect, Google is punishing these content-owners for challenging Google's use of their content in its service.

Luc• 15 years ago

This is not a case of technical ignorance, though it's hilarious to pretend it is. The French-language newspaper association wants to receive money from Google (for the ads displayed next to their content) and figured it would be easier to pressure them through the courts than through other means.

ToadyCensored6• 15 years ago

The lawsuit was over the use of copyrighted excerpts from articles in the Google News aggregator and cache, such as lead paragraphs.

Google is trying to pretend that the court ruled all hyperlinks to be illegal. Google was defiant before over this ruling when it ignored an order to publish the results of the verdict on its website, so it's not far-fetched to believe Google is once again throwing its weight around and reminding people of the power it wields over the livelihoods of content creators.

franze• 15 years ago

   User-agent: Googlebot
   Disallow:

   User-agent: Googlebot-News
   Disallow: /

Bootvis• 15 years ago

The papers are back according to this news article:

http://babelfish.yahoo.com/translate_url?doit=done&tt=ur...

They also hint that the complaints have been lifted by the papers.

feb• 15 years ago

Yes, they are back in the index: http://www.pcmag.com/article2/0,2817,2388635,00.asp

And in a simple test, it looks to work too: http://www.google.com/search?q=copiepress+lalibre+critique The second link (lesoir.be) is one of the newspapers.

Belgian newspapers: Google blocking us on searches

Related Stories