Microsoft Will Soon Start Charging For Its Bing Search API

http://techcrunch.com/2012/04/12/microsoft-will-soon-start-charging-for-its-bing-search-api/

by motti_s • 14 years ago

84 43 14 years ago

techcrunch.com

rlpb• 14 years ago

Are there any implications for DDG here?

notatoad• 14 years ago

i believe that DDG accesses the Bing indices through the Yahoo BOSS API, and yahoo already pays microsoft.

eli• 14 years ago

That does not seem like a stable long-term arrangement, but what do I know.

mda• 14 years ago

Does anybody know what percentage of queries in DDG site is answered by bing?

guimarin4• 14 years ago

so long as you're not doing a !bang specialized query, all of them.

tcwc• 14 years ago

This happened a few years ago when Yahoo BOSS started charging, back then Gabriel said they'd moved over to Bing (http://news.ycombinator.com/item?id=1612081).

I'm sure they still make heavy use of the Bing API, but have since expanded their range of sources to soften the blow somewhat. They're now getting results from Blekko (who run their own index), and are I'm sure they've been building out their own index. There's a full list of sources here - http://help.duckduckgo.com/customer/portal/articles/216399-s....

We also don't have any pricing details for higher volume usage of Bing, and DDG are in a much better position to negotiate a better deal these days.

RyanMcGreal• 14 years ago

Probably not a bad idea on balance. They're not going to capture market share in any case, so this may be the only way they can recover some revenue.

cannuk• 14 years ago

This seems like an odd move. It's not like bing has any traction with developers at all. Wouldn't charging them make it even harder to gain traction? I am not familiar with thei API, what does it have over google that would make me pay for it?

sycr2• 14 years ago

Google doesn't have a search API (it was long ago deprecated https://developers.google.com/web-search/). So the API itself is the advantage over Google in this case.

buu7002• 14 years ago

Did you read the link you provided? They actually mentioned right at the top that they have a newer search API which is recommended (though I'll admit that I missed it too at first).

https://developers.google.com/custom-search/v1/overview

guimarin4• 14 years ago

No surprise there. I wonder if all the faux search engines will have to start either crawling/indexing, or transition to Knowledge Engines ( I'm looking at you DDG ). Curious to see if this sparks people to start more search companies.

dewitt• 14 years ago

Having been through similar decision making processes myself (with the Google Custom Search API, the Google Translate API, etc), this is just as likely an abuse mitigation technique as it is a revenue generation opportunity.

Requiring even a modest at-cost fee for a web service does wonders to discourage all sorts of misuse, from wanton large-scale data mining, to blatant repacking and resale, to worse. (Heck, simply requiring a valid credit card alone helps.)

And sadly no, simply having low quotas for free access doesn't entirely suffice. If there's material value to be extracted from a free service, you'd be amazed at the lengths people will go through to create large numbers of low-volume scrapers. Most of these are obvious and easy to detect and defeat, but continually doing so adds up in cost, and it takes engineers away from providing better services to legitimate customers.

In short, most people on the outside don't appreciate just how difficult the handful of bad guys make it for companies to do something good for the other 99%. So I'm sympathetic to Microsoft here, I really am.

Aaronontheweb• 14 years ago

This. The best way to fight these types spammers and scrapers is through economics - provide the content at-cost and it no longer becomes cost-effective for pray-and-spray models.

Imagine what the world would be like if there was a cost-per-unit to sending email - our inboxes would be a much saner, friendlier place. Fewer marketing emails, virtually no spam (no longer economical).

gl0wa• 14 years ago

Requiring valid credit card may not by enough - it's still possible to walk around it by generating lots of virtual credit cards, even on the fly.

denzil_correa• 14 years ago

Yeah! I will use it.

harryf• 14 years ago

This could be a good thing. There's a niche for web search as a service plus paying customers might help steer Microsoft to better things with Bing

guard-of-terra• 14 years ago

This is important because some leading local search engines (seznam, naver (?), baidu) use Bing for their global index.

zaidf2• 14 years ago

This got me thinking...is it possible for Microsoft to secretly buy out DDG and leave them the fuck alone except where they need help(ie. access to Bing index)?

For all the money the Bing unit keeps pouring, I feel buying DDG and leaving them the heck alone can be a reasonable long-term bet with relatively little risk.

The challenge would be to keep the founders/team motivated. They could spin it off as a completely different company on IPO track and give significant equity. But then what if Google wants to buy them out? And of course, the Bing team may have a problem with MSFT creating internal competition though that may just push em to do better.

More likely if DuckDuckGo gets in an acquisition bidding war, I put my money on Gabe passing acquisition for raising a huge funding round that lets him take some money off the table.

With all the flaws(enough that I don't use it), it remains a rare search engine start-up that has its heart in the right place: to actually serve consumers versus build some technology or team and get acquired(looking at you, Powerset).

guimarin4• 14 years ago

I think you're getting ahead of yourself here. DDG has 0.1% of all search traffic ( 30m out of 23b queries per month ). They get all of their 'relevancy' from bing, and blekko. The reason you like them is because they clean up these results, provide peripheral add-ons like user privacy, no ads, easier syntax for power users, and one-boxes which function more as a knowledge-base than a search engine. They don't crawl/index the web. or if they do, we haven't heard anything about it, or seen any different relevancy rankings from the BOSS api. So they are really a new face to already extant search engines.

You mention a 'bidding war'. Who would buy them? Any 'features' they provide can be copied by msft/google if they feel threatened. They don't have their own backend search technology so partners who might want to do search with them, ask.com, search.com, etc. have no reason to work with them as opposed to the BOSS api themselves or even Bing. I'm just as excited as you are about innovation in search and competitors to google, but DDG needs to be re-architected on the back-end before these sorts of pronouncements make sense.

AznHisoka3• 14 years ago

yep, i completely agree. DDG is built off Bing's platform, so they're screwed if MSFT ever made their index private. They also don't got a strong brand outside of hackers who've heard of them.

I must admit they do a good job in the PR area of making themselves seem big and sexy. But they're really not innovating search, or providing any thick value in improving search. At least Blekko was trying to create a better algorithm.

dwd• 14 years ago

DDG has its own crawler that suppliments and cleans the main search results.

http://www.gabrielweinberg.com/blog/2010/07/google-web-spam....

DuckDuckBot

http://duckduckgo.com/duckduckbot.html

motti_s• 14 years ago

Here is something ironic. I bet you many of DDG's users are hackers who use it for privacy reasons. The same hackers who rely on Google Analytics in their own websites. If DuckDuckGo grows to become more than a niche search engine, the same hackers who use it will have to reinvent Analytics somehow. This is reason #1 why DDG will stay small. Reason #2 is that if necessary, a DuckDuckGoogle can be created in an afternoon's worth of effort in Mountain View.

zaidf2• 14 years ago

Reason #2 is that if necessary, a DuckDuckGoogle can be created in an afternoon's worth of effort in Mountain View.

I can't tell how serious you are with that statement.

eurleif• 14 years ago

I don't understand your line of reasoning. Yes, Google Analytics is useful, and we would have to replace it if Google the company went out of business or discontinued it, but why is that going to motivate people to use Google the search engine?

Aloisius2• 14 years ago

Time for people to start supporting something like Common Crawl http://commoncrawl.org/ to build their own search engines.

CurtHagenlocher• 14 years ago

Because by contrast, that will be free?

joedevon• 14 years ago

It is free. And will continue to be so. It's not-for-profit.

fungi• 14 years ago

im rooting for http://yacy.net/en/

love the idea but search results are balls (ATM)

greglindahl• 14 years ago

blekko has been running a crawl+index of several billion pages for 2 years now, so perhaps I can talk about this a little.

If you want access to a big crawl to grep through it for interesting data, then Common Crawl is awesome and inexpensive and I don't think you can get anything like it for the price, unless your query is simple enough to run as a blekko webgrep (https://blekko.com/webgrep).

If you want to build a search engine, Common Crawl isn't so useful. Search engines want _directed_ crawling of the pages that they think are good. Crawling is only a small fraction of the total work done in a search engine. Search engines generally aren't on AWS, because the right configuration of machine isn't rented by Amazon -- serving queries needs SSDs or more ram and less cpu than what Amazon offers. So, what Common Crawl offers a search engine is higher costs and mostly bad data.

Aloisius2• 14 years ago

I believe Common Crawl will do a directed crawl if you contact them.

loverobots• 14 years ago

Why Microsoft? Do you really need that $2 per 1000 or whatever when you could gain adoption?

caryme• 14 years ago

In my experience with the Bing API in the last several months, I've found that you get what you pay for. Its performance has been inconsistent at best, to the point which I created the site http://isthebingapiworking.heroku.com/. The web search api frequently orders of magnitude fewer results than the actual website, a problem making a frequent theme in their developer forums http://www.bing.com/community/developer/f/12254.aspx.

By charging for their search API, I would just say that Microsoft is beginning to take their API seriously. It seems pretty clear that minimal resources, if any, were dedicated to the free version.

I'm bummed, because I've found relative success using their news search api (particularly for the article aggregation component of http://www.congressionalprimaries.org/), and now we'll have to look into alternatives, but if this means actually providing a decent product, I think this is a good move for Microsoft.

ww520• 14 years ago

Actually the search API would be interesting for domain specific search. You can use the API to create a site to present result specific to MP3, for example, formatting the result with the MP3 attributes.

zmonkeyz• 14 years ago

Maybe Megadeath can make a "Bing sells, but who's buying?" album.

Microsoft Will Soon Start Charging For Its Bing Search API

Related Stories