An Interview with DuckDuckGo's founder, Gabriel Weinberg

http://www.techspot.com/article/559-gabriel-weinberg-interview/

by marioestrada • 14 years ago

108 38 14 years ago

dkhenry2• 14 years ago

I like DDG and I even converted chrome over to use it as its primary engine, however, recently I am noticing more and more spam links and less and less links that really matter. I think I am coming to the conclusion that DDG did this is the wrong order. They are developing a great frontend and using others backends, but to really beat google or MS or yahoo you need to develop a great backend that can filter results well and troll the web efficiently then you can put an awesome frontend on it.

I still think the real silver bullet will be to make a backend system that can peer with other systems to gather data and can be customized to get deep results from a small subset of the web that interests a particular user ( think corporation uses it for internal search and to make their site show up better on a common front end). I even started a research project to begin working with some of the things needed to accomplish it[1].

1. https://github.com/dkhenry/SimpleMapReduce

Ralith• 14 years ago

> you need to develop a great backend that can filter results well and troll the web

Trawl. The web gets trolled enough as it is.

Steko3• 14 years ago

Not to hijack further but 'troll' is technically fine there.

troll ... 4.(intransitive, fishing, by extension) To fish using a line and bait or lures trailed behind a boat similarly to trawling; to lure fish with bait. [from circa 1600]

http://en.wiktionary.org/wiki/troll#Verb

vaksel• 14 years ago

yeah you'd think with the VC investment they'd be going for that

boyter4• 14 years ago

I would imagine there would be a few problems with that which might be insurmountable.

The first being speed. People complain about DDG's speed already. Relying on a host of external searches would only cause more issues. Especially if as Google reports that a huge amount of queries are unique.

Another would be ranking. Who determines what is the most relevant result to a query? While you have multiple sources knowing which one to go to has to be determined somewhere. If you hand it over to the peers, they can game the system by insisting they are the most relevant. If you leave it to the server what incentive do the peers have to participate? If you use an open algorithm which people use whats to stop people from gaming the system?

I don't think there is an easy answer to the search game. You either need to build on others systems to have something compelling, or have millions (if not billions) of cash to have enough runway to build/improve your own system, improve it to the point its worth using and then turn a profit.

I don't even think its the complexity of the problem that stops the second. It's bandwidth and disk storage. My personal prediction is once we get disk's up-to a size where you can store a sizable chunk of the web, coupled with enough bandwidth to crawl it in a reasonable time you will see more innovation in the search space as the barrier to entry will be lowered.

mmahemoff2• 14 years ago

"Especially if as Google reports that a huge amount of queries are unique."

Google reports 20-25% of queries are unique, way more than I think most of us would expect. http://www.readwriteweb.com/archives/udi_manber_search_is_a_...

We can probably assume it's not evenly distributed across people. Some people would probably always get cached queries, while others would need fresh results at least half the time.

Mythbusters• 14 years ago

This is one product I hope people really get behind. Good interface and smart aggregation. Keep it up guys!

MikeCapone5• 14 years ago

I've been using DDG as my default browser for a while (though I sometimes revert to !g to search google), and I'm satisfied with the experience.

The main thing that would really improve my experience would be if it was faster. Google really spoiled my with the instant results and suggestions as you type.

I'm also looking forward to them switching to SPDY as I always use the encrypted version, and this should make it a bit more responsive.

Steko3• 14 years ago

Google has put so much effort into speed over the last 10 years that they can't perceptively improve much in this department going forward. So DDG's target is relatively stationary and closing the gap may be expensive but is at least straightforward.

MikeCapone5• 14 years ago

That's a good point, though I'm afraid it might be one of those things where diminishing returns mean that it becomes very very expensive to catch up, and I don't know if DDG has the resources. Moore's Law should help, though, but personally I haven't really felt (subjectively) like DDG has become much faster since I've been using it.

mmahemoff2• 14 years ago

Well, it's only stationary if Google's features are stationary too. e.g. when they introduced live search, that's a new performance target any competitor has to reach.

JD557• 14 years ago

One of the main reasons I changed to DDG was because I was having some problems with google:

- Some times the instant search simply doesn't work until I press enter (and sometimes, even when I do, the results just don't appear) - I noticed that google doesn't take you directly to the page. It takes you first to an intermediate page (probably for tracking purposes) and then redirects you to your result. I would be fine with that, but sometimes my browser just got stuck in the intermediate page (no idea why).

I sometimes need to use !g too and I agree that google sometimes is faster than DDG (at least when it works). What I really miss from google is the autocomplete

MikeCapone5• 14 years ago

Agreed. Autocomplete would be awesome. I think it's pretty much expected from any search engine nowadays. I like how Chrome and iCab will sometimes use google's autocomplete even if I'm doing a DDG search, but I wish it was also available straight from the site's search field.

brokenparser• 14 years ago

Error: DDG is not a browser.

MikeCapone5• 14 years ago

I meant 'search engine', that was a weird typo. Sometimes my fingers don't listen to my brain.

AtTheLast• 14 years ago

I like what I've seen from DDG so far. I don't expect it to be Google, but it seems like the search engine keeps getting better with each iteration.

With the launch of DuckDuckHack it will be interesting to see what people build for the platform. Plus, I'm just excited to see a talented team take on search.

isaacwaller• 14 years ago

I have never liked DuckDuckGo after seeing their advertisement tactics, especially the "educational" http://donttrack.us/ minisite. Apparently your anonymous HTTP referer info (from Google Analytics?) will be sold to insurance companies / appear in a background check? It is blatant fear mongering and really made me angry.

stephengillie• 14 years ago

It all goes into the huge (secret by law) national insurance database, which is accessible by all insurers. All information gained by an insurance company about an individual is put in there, and insurers are prohibited from discussing it.

Jayschwa• 14 years ago

Citation(s)?

toomuchcoffee• 14 years ago

http://www.techspot.com/news/49975-how-ants-have-used-the-in...

If you look at http://donttrack.us/, it says your data "can potentially end up" in those places, not "will be sold to..."

Reply

raghus• 14 years ago

The article mentions $115K in revenue last year. How is that close to self-sustainable for a 10-15 person company?

Reply

dkhenry2• 14 years ago

They average 500% growth. If revenue also scales like that then they are well on their way to profitability.

Reply

603techguy• 14 years ago

That doesn't explain sustainability now.

Reply

JohnsonB• 14 years ago

DuckDuckGo has really fallen behind Google in the zero click information department, which is one of DDG's key selling points. DDG needs to have one or more major competitive advantages over Google in order grow. Privacy is great, but zero click is something obvious and usability orientated that really stands out. I'm not going to use DDG if Google gives me more zero click information no matter what; it just makes too significant a difference in search experience.

Reply

boyter4• 14 years ago

What about things like,

  https://duckduckgo.com/?q=frequency+of+letters+in+The+quick+brown+fox+jumps+over+the+lazy+dog
  https://duckduckgo.com/?q=days+between+6%2F22%2F1979+and+10%2F5%2F1979
  https://duckduckgo.com/?q=hn+duckduckgo+interview
  https://duckduckgo.com/?q=php+xml_parser_create+example
  https://duckduckgo.com/?q=currently+in+theaters
  https://duckduckgo.com/?q=msft
  https://duckduckgo.com/?q=currency+in+panama

The only one I think Google does better is "currency in panama" however it also gets the information wrong in the "zero click" answer. The only reason I like that result more is the Wikipedia answer on the right is just more appealing to my eye.

Reply

comex• 14 years ago

Careful... for "hn duckduckgo interview", at the moment, DuckDuckGo shows a HNSearch widget with three (stale) results, none of which is this page; no relevant results appear in the main listing. Google has no special widget, but this page comes up first in the main listing.

To digress a bit, zero-click is great when the information you want is actually accessible with zero clicks, but it's very, very limited: as soon as you need to click through to a website, special widgets can't compete with a solid backend for regular search results. That's why I can hardly imagine switching to DuckDuckGo...

Reply

whichdan• 14 years ago

I used DDG for a few weeks, but ultimately switched back to Google. It was mainly for two things:

1) It's slightly slower than Google, which became more apparent after the fifth or sixth search of the day.

2) No images integrated with search results. I didn't realize how often I searched for images until I used DDG. At Google I usually get a few images and an "Images" link to click. At DDG I needed to add a !gi to my query.

Like JohnsonB said, there really need to be another draw besides just privacy.

Reply

panacea• 14 years ago

I've found I add !gi to my search string subconsciously nowadays, when I know I want to perform an image search, whereas before I would enter my search terms, wait until google had loaded its results and then hunt for the images link to get the full image search results.

In some way it's 'trained' me to perform more targeted searches from the outset.

The slight lag before getting results is noticeable though.

I'm going to stick with the upstart for the moment nevertheless.

Reply

mark_l_watson• 14 years ago

I purposefully use duckduckgo several times a year as my default search engine for several weeks because I like to support alternatives to mega corps.

Google meets my needs better and their recent power search class was very cool.

A little off topic but I have had more than a few fantasies about starting my own micro search engine. Text analytics and knowledge management in general have been an interest of mine since the early 1980s. What stops me is that if I were to invest my personal resources in this I would want tens of thousands of users getting value from my system every day, and frankly, I don't think I could achieve that. Gabriel gets an order of magnitude more than what I would hope for, so I hope that he is very satisfied with what he has achieved. Good job!

Reply

pacomerh• 14 years ago

Also, don't miss Gabriel's interview on Techzing http://techzinglive.com/page/423/techzing-68-gabriel-weinber...

Reply

petdance• 14 years ago

Sad that there's no mention of the huge role Perl + CPAN play in their infrastructure:

* http://help.duckduckgo.com/customer/portal/articles/216392-a... * https://github.com/duckduckgo/duckduckgo/wiki/DuckDuckGoPerl * http://www.perlmonks.org/?node_id=848999

They've also been visible in sponsoring the Perl community.

Reply

An Interview with DuckDuckGo's founder, Gabriel Weinberg

Related Stories