DuckDuckGo browser seemingly sends domains a user visits to DDG servers

https://github.com/duckduckgo/Android/issues/527

824 468 6 years ago

tananaev3• 6 years ago

Very weak argument for why they do it. Using a service to retrieve a favicon? Surely there's a way to implement the same logic locally.

heavyset_go3• 6 years ago

BitWarden does this, too.

slenk2• 6 years ago

If you host your own Bitwarden server you don't have to worry about that.

Also different expectations

toyg2• 6 years ago

BW already has all the URLs of services you use: you saved them yourself for them to keep safe. It's obvious you already trust them enough to know that sort of data.

ghewgill• 6 years ago

So if favicons were gathered locally, then you would prefer that your own browser would reach out and make multiple requests to every (potentially dodgy) site listed on the search result page?

Note also that these are favicons for results that DDG has already given you. This isn't tracking your clicks. The list of sites that appear on the search result page is not new information to the search engine that just gave them to you.

EDIT: Like other commenters, I was not previously aware that DDG had a browser, and my comments were about this behaviour for the search engine results page.

tananaev3• 6 years ago

Are we talking about search results or sites you visit?

detaro2• 6 years ago

What about the issue linked suggests to you that it is about search results in any way?

colecf• 6 years ago

Can you explain your edit a little more? I still only understand your original viewpoint. Just because there's an app doesn't mean they don't have to contact DDG servers at all, right?

tagawa3• 6 years ago

We had already had created this anonymous favicon service for our private search engine. In addition, doing it this way avoids another request (and potentially multiple) to the end site.

The service is private as we do not collect any personal information (e.g. IP addresses) on any requests for this or any service and the requests are all end-to-end encrypted.

systemvoltage2• 6 years ago

Why not do it locally? It is one extra request to the favicon link in the header and cache it. Or cache it when the user visits the website. Or explicit button in the settings UI to refresh favicons locally.

There are so many options.

haggy2• 6 years ago

Your answer to all these criticisms is a lazy one. You are saying "this already exists for other things so we use it regardless of what the correct method is". Just because something exists in one form doesn't mean it's the correct choice.

oska• 6 years ago

Please don't downvote this reply from a DDG staff member (as I'm currently seeing).

Even if you don't like the reply it's good that we're getting replies.

aronpye3• 6 years ago

Good privacy is not based on trust. Even so, this incident is even less reason to trust you.

jb7752• 6 years ago

I personally believe you, but this approach adds multiple unnecessary trust boundaries. Just thinking about how many separate logs are triggered outside of DDG is troublesome. And what happens when you guys think everything is secure, but one day find out it isn't?

I understand utilizing a single service/feature to accommodate multiple platforms is a big win from a programming standpoint. But if it carries the risk of losing the trust of your user-base while at the same time risks breaking the core mission statement of the business (privacy), it's probably not worth it.

SahAssar• 6 years ago

What do you mean by "end-to-end encrypted"? Isn't this just a request to your service that will then go out and fetch the icon from the site, so it can't be e2e since you need to know which site to request the icon from.

Why use that term when it clearly can't be true or even seems applicable/relevant?

marcinzm5• 6 years ago

This is concerning because it indicates a lack of care in terms of privacy and understanding that the best privacy is achieved by knowing the least. Does this approach permeate their backend as well?

Stevvo• 6 years ago

A privacy focused browser shouldn't even have a backend.

fulafel• 6 years ago

I'm not so sure. I think the privacy-functionality trade-off is understood and expected by the users, it woul be used by very few people if it were extremely spartan.

(But this instance, the favicon service, is not a good privacy-functionality trade-off)

markholmes• 6 years ago

Does this only occur in DuckDuckGo’s Android browser?

jakejarvis2• 6 years ago

And the iOS browser it seems: https://github.com/duckduckgo/iOS/blob/1ae03d7221180bd6791cf...

BuckRogers• 6 years ago

Essentially a way to collect where people are visiting. I believe them that it’s anonymous, this valuable info wouldn’t need to be identifiable to be of value.

They should probably change the behavior to how it’s suggested in the thread, but I’m still going to use DDG over alternatives for the bang feature.

coronadisaster2• 6 years ago

I don't really trust DuckDuckGo, but I use their search service because I trust Google less... I still trust Firefox more for a browser although it won't take much at this point to make me switch.

projomni3• 6 years ago

a paranoid android-:)

coronadisaster2• 6 years ago

yeah Android is a lot worst than using Google search

kovac• 6 years ago

There's Qwant, a search engine. I'm using that. It's not as fast as Bing, but it does the job and seem to be trustworthy.

stan_rogers2• 6 years ago

There's always Startpage. You use Startpage; Startpage uses Google for you, with their own user agent - no history, no tracking.

Nextgrid3• 6 years ago

How does that work?

What does Google have to gain from it? Google has pretty aggressive anti-scraping protections to protect against this exact behavior, so why would they allow Startpage to get away with it?

What does Startpage have to gain from it? Unlike DDG, they don't seem to have any core product, so they fully depend on Google's goodwill which is very shaky grounds when it comes to a long-term business.

chris_f• 6 years ago

Runnaroo is also a search engine that is Google based (for organic results). IMO it has an advantage over Startpage because of its integration of vertical search sources, like Stack Overflow [0]. I'm the creator.

Mojeek [1] is also a good privacy option that doesn't get talked about as much as DDG. They distinguish themselves by having their own search index.

[0] https://www.runnaroo.com/search?term=python+unit+testing

[1] https://www.mojeek.com/

moonchild2• 6 years ago

Startpage was bought by an ad company.

bad_user8• 6 years ago

Google is a big target for law enforcement though. The EU for example can't wait to slap them with another huge fine.

Go here and turn your "web and app activity", your "location history" and "ad personalisation" off, if they aren't already:

https://myaccount.google.com/data-and-personalization

If you do that, there's no evidence that Google is more privacy invading than DuckDuckGo, since you're left with taking their word for it. And frankly I trust a big, bureaucratic company more than I trust a startup. The ideal would be to trust technology (e.g. end to end encryption, open source) but that's not the case here.

Note that I also use DuckDuckGo in my Private Mode, as Firefox allows setting different search engine for Private Mode. I do that because it's better to compartmentalize your online personas, plus I keep my "web and app activity" in Google on, with deletion after 3 months.

lopmotr• 6 years ago

Nevermind privacy. How are favicons so complicated that they need a special service that understands edge cases. Just do it one standard way and if a minority of websites don't work, then exclude them. We've been through this mess before with all kinds of web standards devolving into mess.

fiddlerwoaroof5• 6 years ago

The standard way involves a meta tag, right?

jakejarvis2• 6 years ago

That's my understanding of how it's worked for decades...

1. Check for <link rel="icon" ...> tag(s)

2. Check for /favicon.ico

3. ...give up?

Someone correct me if I'm wrong!

aronpye3• 6 years ago

Agreed, creating work arounds for non-standards compliant websites just eliminates the motivation and incentives to be standards compliant in the first place.

CivBase3• 6 years ago

This is a bad look for a company that is trying to build its brand on privacy and trust. Even though I don't use the DDG browser I hope they own up to this, rectify it quickly, and learn from it.

projomni3• 6 years ago

well... i'd much rather fix their deep search so it's as good as google's and not worry about this like this:)

haggy2• 6 years ago

Privacy has to be scrutized constantly and be the top priority. If it isn't, then you're going to end up with another google.

A direct correlation exists between the revenue Google receives for selling data and the quality of its search. Google focuses completely on tracking and search, with privacy behind a far far away afterthought (if it's a thought at all).

dabbernaught4204• 6 years ago

In my view, anyone who trusts ddg is a bit silly - founder has a bad track record on user privacy. Founded Names Database[1], a social media website designed to collect user information as aggressively as possible, before selling all the information to classmates.com.

[1]https://en.wikipedia.org/wiki/Names_Database

Torn• 6 years ago

https://en.wikipedia.org/wiki/Ad_hominem

ViViDboarder3• 6 years ago

Still a better track record than Google, so...

abc-xyz4• 6 years ago

Also worth mentioning they're closed-source, US-based and for-profit. Why exactly do people trust them? Simply because they write a few articles/ads saying "privacy is important"?

If you're willing to sacrifice search quality for privacy, as in switching from Google to DuckDuckGo, then you might as well take a step further and switch from Google to Searx/Ask.Moe.

slmjkdbtl• 6 years ago

most of us use it because we like ducks

akent• 6 years ago

Do they release the source of the webservice? Seemingly not. This is extremely shady.

arp242• 6 years ago

Would it matter? I can release all the source I want, but what guarantees this is also what's running on the actual endpoint?

bangonkeyboard5• 6 years ago

EDIT: I didn't notice that this topic was about the DDG browser (which I didn't know existed) and responded assuming this was about the site/extension. For a browser, yes, a client-side solution is possible and probably preferable. Please check and upvote other comment trees.

This makes sense to me and is not alarming. Getting favicons actually is difficult to do robustly; many applications and websites use Google's service to do so, which then leaks the request to Google: https://www.google.com/s2/favicons?domain=ycombinator.com

Putting this logic in the client is not feasible. You want to send requests directly to every shady site that shows up in your search results, load their pages in the background, work through network delays and HTTP errors, and parse out the location/format of the favicon files?

DuckDuckGo hosting this functionality themselves is also a positive. They have previously been burned when the Web of Trust service they were originally using was found to be farming data, and turned it off immediately once discovered. Processing, hosting, and serving the icon themselves prevents that from happening again.

This is not to say that DDG is perfect: links you click do seem to be redirected through a /l/ page on their domain, which can cause problems: https://lapcatsoftware.com/articles/duckduckgo.html

harry83• 6 years ago

Getting favicons actually is difficult to do robustly

So if you're a privacy browser. Don't. Favicons are not essential. Or, at the very least make them "opt in" and explain it.

projomni3• 6 years ago

Geez, overreacting.

bangonkeyboard5• 6 years ago

I don't see how this is a privacy violation in any way. The headline is that DDG is tracking the domains you visit, but favicons are loaded for every search result you see unconditional on whether you click them, and DDG already knows what those results are when they serve them to you.

stefan_• 6 years ago

I don't need the favicon for a domain I never visited just because it adds some visual cool to your UI. That is not a thing. Cache it whenever I end up visiting the page and be done with it.

bangonkeyboard5• 6 years ago

Cache it whenever I end up visiting the page and be done with it.

DuckDuckGo can't do that because they don't have access to your browser's history and favicon cache, precisely to protect your privacy.

EDIT: This reply is wrong if DDG is the browser.

teej• 6 years ago

The favicon is an important visual landmark that aids in finding the correct search result in the list.

fiddlerwoaroof5• 6 years ago

Don’t they already have access to all the search results anyways? This doesn’t seem to be any privacy loss relative to that.

zekrioca• 6 years ago

This is about their browser, not their search service.

tananaev3• 6 years ago

For the search results DDG can do whatever they way. As far as I understand from the title, this is for the pages that you actually visit. For those pages it totally make sense for the browser to figure out favicon locally on its own.

As for complexity, I'm sure users of privacy oriented software would prefer to not have favicon in small percentage of corner cases rather than leaking browsing history.

dmitrygr• 6 years ago

Putting this logic in the client is not feasible

Wait, what? Why?

rightbyte• 6 years ago

They have to be lying?

detaro2• 6 years ago

Which other browser doesn't fetch favicons locally?

And I have no clue how search results come into this? Of course the browser isn't fetching favicons on search result pages, that's the servers job, but that's not what this is about.

It's really odd that this is the top comment right now, given that even the headline makes it clear its about the browser, not the search results.

mdoms• 6 years ago

Putting this logic in the client is not feasible. You want to send requests directly to every shady site that shows up in your search results, load their pages in the background, work through network delays and HTTP errors, and parse out the location/format of the favicon files?

I don't see why not. Browsers are constantly dealing with these very issues. It's one of their core competencies.

pwdisswordfish25• 6 years ago

"This is not to say that DDG is perfect: links you click do seem to be redirected through a /l/ page on their domain ..."

I am surprised the user is not complaining about this instead of the favicons. Their privacy policy goes on about the privacy implications of Referer headers and instead of calling out browsers for sending Referer by default, they instead give themselves power to record all the user's clicked results themselves. The Referer problem is something that can be solved by the user at the browser level through, e.g., using a client that does not send Referer, browser extensions/plug-ins that can control headers sent, or perhaps with a local proxy to remove the Referer header.

Unless DDG has changed, these prefixed result URLs are the default. It is possible to get unprefixed result URLs using the "lite" version of DDG however that is not the default. "Privacy-focused" search engine chooses less private default. News at 11.

I recently noticed that DDG has started redirecting queries submitted via POST to /lite/. The redirect is to the same domain. No explanation. I have a custom client that does not follow redirects and I now have to submit two sets of HTTP headers instead of one.

These guys are trying to make money from advertising just like everyone else. They have to be very particular in the methods they use to do it -- check the exceptions in their privacy policy -- but it is the same game. Ads and affiliate links. That sort of business and privacy are always going to be at odds with each other.

couchand2• 6 years ago

Unable to reproduce. Links appear to be plain links and no JS intercepts clicks.

bscphil2• 6 years ago

This is done only for browsers that don't support setting the referrer policy. https://caniuse.com/referrer-policy. It should not be happening with any modern browser, and (like the other reply) I don't see it in mine (Firefox).

See here: https://help.duckduckgo.com/results/rduckduckgocom/

richdougherty• 6 years ago

Putting this logic in the client is not feasible.
You want to send requests directly to every shady site that shows up in your search results, load their pages in the background, work through network delays and HTTP errors, and parse out the location/format of the favicon files?

Looking at DuckDuckGo search results and visiting a page that you navigate to are two different things.

1. DuckDuckGo search results:

DDG already returns the search results so there's no privacy violation to return the favicon or the URL for the favicon in the list.

2. Any page that isn't a DDG search results page:

Use client side logic to locate the favicon. This means worse performance but better privacy - which aligns with DDG's goals.

If you want to optimise this then DDG could send the client some precalculated Bloom filters with info about known sites. The client could use these to try certain methods of favicon retrieval first.

rygh• 6 years ago

Don't think it's air tight. But still better than most browsers.

rickyc091• 6 years ago

Looks like this was an issue posted in 2019. From the looks of it, the code remains unchanged.

https://github.com/duckduckgo/Android/blob/b2131d7d2f47fb09d...

niftylettuce11• 6 years ago

Formerly worked with DuckDuckGo

My advice:

Install ungoogled-chromium: https://github.com/Eloston/ungoogled-chromium

Install these extensions: https://github.com/gorhill/uBlock https://github.com/ilGur1132/Smart-HTTPS

There is also a Chromium extension that lets you install from Chrome Web Store: https://github.com/NeverDecaf/chromium-web-store

Set duckduckgo.com as your default search engine with a blank home page. But you could also use @pkrumins home pages of https://techurls.com or https://finurls.com as nice home pages.

Use Mullvad VPN: https://mullvad.net/ (They are EVEN available on F-Droid now, which is AMAZING)

Security harden your Android device: https://niftylettuce.com/posts/google-free-android-setup/

Security harden your Mac: https://gist.github.com/niftylettuce/39597a7b3bc0660ffe1e09d...

P.S. If you need email forwarding for your domain name, you can use something I made. https://forwardemail.net - it is 100% open source.

Follow me @niftylettuce on GitHub and Twitter for more

RandomBacon• 6 years ago

Why ungoogled-chromium over Firefox?

_5vzs• 6 years ago

Faster + less bloat

fastball3• 6 years ago

Why ungoogled-chromium or Firefox over Brave?

Brave is the only browser that includes strong ad and tracker blocking by default, which is the best way to ensure your privacy on the internet.

In fact, every decision I've seen from them makes me think they care the most about user privacy, period.

morpheuskafka• 6 years ago

ForwardEmail is a great service, thanks for making it!

niftylettuce11• 6 years ago

Thanks! I'm rolling out the following within probably the next 7 days:

- All new load balanced infrastructure - Browser extension + API wrapper - Support for pixel tracking blocker (opt-in checkbox or TXT setting) - Smart alerting - Globby/regex support

If you follow my Twitter or the GitHub releases you will get updates.

adjagu• 6 years ago

Could you please elaborate as to why you suggest ilGur1132's Smart-HTTPS instead of EFF's HTTPS-Everywhere?

niftylettuce11• 6 years ago

https://reddit.com/r/privacytoolsIO/comments/5qnq6j/time_to_...

the_duke• 6 years ago

Seems a bit off-topic for the concrete issue.

Advertising your Twitter for the advice of "switch to somewhat well-known browser X, install these very common extensions and use a VPN" is also a bit ... odd.

niftylettuce11• 6 years ago

Privacy-focused web browsers; alternative to DDG browser due to this issue among others.

austhrow743• 6 years ago

An ex DuckDuckGo employee recommending people use an alternative browser over the DuckDuckGo browser, in a post about the DuckDuckGo browser spying on its users, is about as on topic as you can get; after the current employee giving an explanation.

livre2• 6 years ago

Security harden your Android device: https://niftylettuce.com/posts/google-free-android-setup/

I haven't checked all links but some things need to be updated, for example Skimmer Scanner is gone from the Play Store and Yalp Store is abandoned and doesn't work anymore, you should be using Aurora Store. I'd also recommend Aegis or FreeOTP+ over FreeOTP for 2FA. NewPipe is better installed from this[1] repository until this[2] issue is solved.

[1] https://archive.newpipe.net/fdroid/repo/

[2] https://github.com/TeamNewPipe/NewPipe/issues/1981

niftylettuce11• 6 years ago

Yeah, I do use Aurora Store. No longer use Skimmer Scanner. I have to update that post. Working on a new blog/site and a book (eventually). Thanks for heads up.

bluehatbrit• 6 years ago

Your "Security harden your mac" guide has a bunch of amazon affiliate links, it would be good to call that out when piggybacking on a post about privacy at least.

tomtomtom777• 6 years ago

I have a hard time understanding the problem.

The favicon is acquired from DDG servers for the result you've just retrieved from DDG servers.

How is this leaking anything? What additional privacy would you gain from getting the favicons from the domains directly of search results delivered by DDG?

marcinzm5• 6 years ago

This is the DDG mobile browser app, NOT the DDG search engine.

tagawa3• 6 years ago

DuckDuckGo staff here. As mentioned in the linked page, the purpose of the request is to retrieve a website's favicon so that it can be displayed in certain places within the app or on the results page. We use an internal favicon service because it can be complicated to locate a favicon for a website. They can be stored in a variety of locations and in a variety of formats. The service understands these edge cases and simplifies retrieval within our apps and our search engine.

Like our search results, the favicon service adheres to our strict privacy policy[1] in that the requests are anonymous and we do not collect or share any personal information.

[1] https://duckduckgo.com/privacy

fiddlerwoaroof5• 6 years ago

This doesn’t make sense for a browser: just embed the service’s logic in the browser, the browser has all the same information the service could get.

tagawa3• 6 years ago

We had already had created this anonymous favicon service for our private search engine. In addition, doing it this way avoids another request (and potentially multiple) to the end site.

The service is private as we do not collect any personal information (e.g. IP addresses) on any requests for this or any service and the requests are all end-to-end encrypted.

mysterydip• 6 years ago

Are all these edge cases part of the html/w3c/whichever standard? If not, let the edge cases fail. I'm not going to lose sleep over an icon not showing for a site I'm visiting once.

neiman• 6 years ago

I agree. Sometimes it feels that companies throw any possible value out of the bus in order for the perfect UX.

SimeVidas• 6 years ago

How can users turn this off?

lvturner• 6 years ago

Route icons.duckduckgo.com to null.

Sidenote: The more I use pi-hole the more I realise how essential it is!

ancarda• 6 years ago

https://duckduckgo.com/settings#appearance

Uncheck the very last option "Site Icons"

ajnin• 6 years ago

You really can't use "we promise we won't misuse the information" as an argument, that's what everyone says whether it's true or not, and the whole point of using a privacy-centric browser is that as a user you can't trust those kinds of promises.

TedDoesntTalk• 6 years ago

You really can't use "we promise we won't misuse the information" as an argument

Sure they can. Doesn’t mean you have to believe them.

baddox2• 6 years ago

They’re primarily a search engine, so yes, you definitely have to trust that they won’t misuse information about what search terms you are querying for.

lowwave• 6 years ago

By the way how DDG spend money on the ads like at bus stop, I already starting to NOT trust them. Spend the money on development of development of products.

Seems like time to get SearX a try now: https://searx.me

scoutt2• 6 years ago

AND being so reluctant to remove this pinging mechanism adds more doubts about the real purpose.

ddevault6• 6 years ago

I'll state this in no uncertain terms: this is not acceptable, and you need to stop doing it. It makes sense on your search engine, but adding it to your web browser is very much over the line.

I have read your explanations in good faith and they don't cut it. This behavior cannot continue. Good privacy promises are not based on trust - they're based on not ever handling private data in the first place. If you don't quickly admit your mistake and roll this back, it will jepoardize your entire brand - and rightfully so. If you believe this behavior is okay, then it demonstrates incompetence; if you don't believe this behavior is okay but do it anyway, it demonstrates malice.

This is the one thing you Should Not Have Done.

NotSammyHagar• 6 years ago

I appreciate you answering, probably knowing you'd face some negative feedback. Saying "we should trust you, it's for a good reason" is what google and everyone else says. You'll be better off if you just end this. The loss of the fav icon is less important than keeping your credibility.

abnercoimbre• 6 years ago

Seconded. Day in and day out we're given empty promises we can never audit -- now it's being done DuckDuckGo staff too?

Thorrez3• 6 years ago

is what google and everyone else says.

Do Chrome, Firefox, or Safari do this? I would assume they do it on-device.

abrookewood• 6 years ago

This is the correct answer. We all remember "Do No Evil" and have been around long enough to see that sentiment die. Don't go down this path.

st3fan5• 6 years ago

Take the code from Firefox iOS or Android-components. We spent a lot of time on these and it is all on device.

https://github.com/mozilla-mobile/android-components

https://github.com/mozilla-mobile/Firefox-iOS

nexuist• 6 years ago

What a world class reply. This is what makes open source valuable.

antpls• 6 years ago

I wonder how many lines of code from big open source applications are generic enough to be reused in other projects.

Firefox and Google Chrome probably have the equivalent of many small high quality libraries embedded in them, implementing 'business' logic or protocols, that could be reused in more places.

I guess a large scale study on github could be done, with a graph analysis to show potential "cut off" points in codebase.

weinzierl• 6 years ago

Yes, apart from Chrome's implementation there is probably no better tested and more mature implementation. Still it seems to be a non-trivial problem because Firefox sometimes shows me a favicon from another site I used.

dTal2• 6 years ago

Why don't they just take the code from their "internal favicon service"? The whole thing smells.

_hl_4• 6 years ago

It's not immediately obvious whether it is more privacy preserving if the client automatically makes a request to each site in the search results while scrolling through the results, especially since you're already trusting DDG when performing the search.

Maybe this should be an opt-in rather than an opt-out feature?

Edit: as pointed out by warpspin in another comment, this is about the DDG Browser, not search results.

ximeng2• 6 years ago

You're repeating what's on that page, which is exactly what everyone is worried about in the first place.

jtdev• 6 years ago

I don’t care if I get the favicon or not... can I just opt out of this functionality?

pschastain3• 6 years ago

From elsewhere in this thread:

https://duckduckgo.com/settings#appearance

Uncheck the very last option "Site Icons"

ViViDboarder3• 6 years ago

Complicated code can run just fine on device.

I’ve been an avid DDG user for years and it worries me that DDG staff don’t see why this is an issue. We shouldn’t have to trust your privacy policy if you minimize exposure.

dTal2• 6 years ago

This. No matter how tricky the logic is, there's literally no reason to run that code on DDG server's and throw in a remote connection. What the hell?

alistproducer2• 6 years ago

The reviews are in for this response and they are bad. It's concerning that given the react it got, there's no edit addressing the concerns. The HN audience has to be the power user, bread and butter of a product like this and when you see a company ignore the concerns of a key constituency like this, their future almost never looks bright.

bluetwo• 6 years ago

Can you tell, for instance, how many of your users visited site A?

Can you tell how many visited site A and also site B?

Mandatum• 6 years ago

What a bizarre potential privacy flaw to introduce for a tiny little icon nobody cares about. I understand it, usability and UX is important - but you guys are DDG! Come on! Your customers all have tinfoil hats!

Aeolun• 6 years ago

Your customers all have tinfoil hats!

Generally speaking. Mine is shielded with lead.

iandanforth• 6 years ago

This is obviously an insufficient answer. Why risk your one selling point on such a trivial bit of code?

sujesh• 6 years ago

come on dude. I thought you got a better answer.

byteshock• 6 years ago

I’m very disappointed in how you guys responded to this. As a privacy focused company I would not have expected an answer that sounds like it came from a data collecting company like Google.

I just switched to DDG browser a week ago and will now be looking for a new browser now. I hope you know this is not an appropriate response to the situation. Especially because all you guys do is preach about how much you protect your users’ privacy. Now you’re here asking us to trust you not to abuse our data and just linking us your privacy policy. I’m sad to say that my faith in the DuckDuckGo company and team is now lost.

minerjoe• 6 years ago

Lost lost lost.

I'ts not like we couldn't have predicted this disaster.

One more reason to never trust a companies "word". Show me the code.

ComodoHacker2• 6 years ago

Technical aspects aside, don't you agree it's a legitimate privacy concern from the user's point of view?

jacquesm7• 6 years ago

Does Gabriel know about this? If not could you please clue him in and get some guidance because you are absolutely getting roasted here and are wrecking DDG's carefully built up reputation. I can easily see how this might seem to be a good idea to you and other DDG engineers but it goes 180 degrees against DDG's stated mission. In other words: you may be well outside your paygrade on this.

Igelau3• 6 years ago

this might seem to be a good idea to you and other DDG engineers

If that's true, then I am so glad they ghosted me when I applied there.

drawfloat• 6 years ago

I don't think it's fair to say this comment is 'wrecking DDG's carefully built up reputation'. If what he describes is damaging their reputation to you, that's on the business, not on the commenter. If anything, the commenter seems to have been honest and straightforward with what's going on.

Sounds like you'd prefer him to have run a message past management/public relations first?

raxxorrax2• 6 years ago

you may be well outside your paygrade on this.

Not as worse as publicly denouncing an honest engineer while referencing his paygrade. I hope there is no affiliation you have with DDG to be honest, because this is much, much worse.

kchoudhu• 6 years ago

This is an asshole response.

im3w1l3• 6 years ago

For me it does the opposite. Hearing engineers speaking off-message inspires trust, while pr-evasion and weasel speak makes me think they have something to hide.

Igelau3• 6 years ago

At this point we're all well aware why the app phones home. Continuing to spout that like it's some ward against the fact that this is a very real vulnerability is an insult. Trust me, your target audience doesn't give a crap if favicons work; they care that DDG acknowledges the risk of a glaringly obvious vulnerability. Who do you even think you're arguing with on HN and GitHub? My children can't multiply yet but they'd be able to understand why this is bad practice.

The repeated handwaving that no one in your company is ever going to do something bad or stupid when the browser phones home for what amounts to a cute sticker is extremely suspicious.

speedgoose• 6 years ago

This is a bit concerning that for a company with a marketing so focus on privacy, you guys thought it was a good idea to have such a service.

Nobody in the company at any point thought that it could be a problem?

Your strict privacy policy mean nothing by the way, because you are a USA company and you must respect your local laws such as the patriot act, which are not very privacy friendly.

bn7t• 6 years ago

Please choose another hill to die on. This is just not worth it. Clearly it's possible to do this on device like mozilla did it.

aronpye3• 6 years ago

The road to hell is paved with good intentions. At the end of the day, your privacy policy is just a bunch of words with nothing to actually prevent you from abusing the data collected. Instead of us relying on DuckDuckGo to act ethically, just don’t collect the data in the first place.

MrGilbert2• 6 years ago

Maybe I'm too old for this, but wasn't a favicon supposed to be located at "fancy.url/favicon.ico", or alternatively as a "<link rel="shortcut icon" \>"?

Curious to know why this is an issue.

ricardo812• 6 years ago

There are variations of it now due to mobile devices, social sharing etc that may prefer a larger icon.

An online favicon generator will create these variations

Nonetheless, the browser can see this when parsing the page and choose the appropriate path.

rezonant2• 6 years ago

It's really not an issue. The rest of the browsers handle this without a web service (even Chrome apparently)

dvdkhlng3• 6 years ago

In germany we have the words "Datensparsamkeit" (data parsimony) and "Datenvermeidung" (data prevention) [1]. Which wikipedia merely translates as "Privacy by design" [2].

DDG is unneccessaryly producing (aggregating), transmitting (and collecting?) very sensitive user data here, which is just the opposite of data protection. I can't even understand why they try to justify their actions. It's like omitting the seat-belt in a car, then telling customers that this was required to make the in-car entertainment system more usable.

[1] https://de.wikipedia.org/wiki/Datenvermeidung_und_Datenspars...

[2] https://en.wikipedia.org/wiki/Privacy_by_design

warpspin8• 6 years ago

In fact I think what they do here is illegal by GDPR. It does not matter that they say they do not collect the information, it is enough it is unnecessarily sent to their servers to make the whole function illegal.

The transmission of ip address alone, which is necessary for the TCP request to happen, deanonymizes the request enough to not be considered anonymous within the GDPR framework.

GDPR Article 5 (1) c: "Personal data shall be ... adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimisation’);" - this is the "Datensparsamkeit" you mentioned.

Exceptions from Article 7 do not apply: The user has to give wilfully give informed consent, which he cannot do as the privacy policy of the browser omits the information that all visited domains are transmitted to DDG servers.

GDPR Recital 30 "Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags."

Oh, and the fact I'm downvoted for a purely informational comment additionally does not shine a good light on DDG.

beshrkayali• 6 years ago

I'm surprised at how you're handling this. DDG is supposed to be friendly to privacy-aware users. You're dismissing people's valid points and asking them to trust you, just like any other privacy-non-friendly service would do.

Edit: I'm speculating here. But specifically because of the way you've replied here and on Github, my actual level of trust in DDG team went down.

rezonant2• 6 years ago

Doubling down is fairly ridiculous when one has to imagine the original reason for doing this was to save on time and engineering for the app by leveraging what DDG had already built, but a mindful response would be that you are aware of the downsides of this approach and you'll be working to change it.

Besides, how do you handle Intranet, VPN sites, and auth-only sites where DDG's god-tier favicon parser in the cloud couldn't fetch the URL anyway?

izietto• 6 years ago

You are just repeating what's already written in the link, it isn't very useful. Try to address users' concerns instead.

zzo38computer2• 6 years ago

Does it have an option to disable the internal favicon service and/or an option to disable favicons entirely? (I disable the favicons on my own computer, since I don't use them.)

leowoo91• 6 years ago

favicon has fairly enough known meta tags in case favicon.ico url is lacking

bluesign4• 6 years ago

Sorry but this is not enough reason. There is a simple question you should ask to yourself.

- Would you be ok to use a third party for this with same privacy policy?

dna_polymerase• 6 years ago

We use an internal favicon service because it can be complicated to locate a favicon for a website

That must be the worst justification for this possible. Favicons. Complicated to locate? Who are you trying to fool, 5 year olds?

gap• 6 years ago

It’s amazing how tone deaf technologists can be when it comes to privacy, even when they have nothing to gain by exploiting the user’s data. DDG’s response reminds me of Mark Shuttleworth’s argument that they “have root”, so we can trust them with our life.

Dear DDG, you are getting complaints on GitHub and Hacker News. This is not the general public, it’s people who understand the issue. You should definitely reconsider whether you’re doing something wrong.

scoutt2• 6 years ago

This answer and other replies from yourself in this page seem copypasted from here: https://help.duckduckgo.com/duckduckgo-help-pages/privacy/fa...

why?

Sir_Substance• 6 years ago

I don't know how you can misunderstand your core demographic this badly mate.

If you think the next time I hit the shitter I'm not going to be looking for a new browser, you're dead wrong.

Just do the basic checks and then fall back to a DDG logo, no one cares that much about the favicon.

bambax• 6 years ago

the purpose of the request...

... is completely irrelevant. Even if they were trying to save babies from a fire (which they really aren't) it wouldn't excuse the fact that they're doing something orthogonal to their stated policy and sole reason for existing.

Everyone makes mistakes, that's not the point. The point is to correct them when they're found, instead of digging one's heels in the ground and pretending it's nothing.

sm2i• 6 years ago

so, essentially you blew privacy because of favicons? favicons??

awinter-py2• 6 years ago

seems like the ticket author found this by reading code (presumably was grepping for duckduckgo.com URLs)

this would never happen with a consumer-facing product from apple or google; someone would have to MITM their whole OS to discover phone-home

LeoNatan25• 6 years ago

It actually happens very often. People monitor their networks often, and pay close attention to such problems. Just search for “Apple phone home” and you’ll find many cases where people complain about Apple’s various services making worrying requests.

awinter-py2• 6 years ago

right sorry -- didn't mean to say that they'd never get caught, rather that you can't read source

TabbyCatKirk• 6 years ago

Everyone is missing the point here. Let me break this down as simple as I can:

1. End user does a DDG search for "food" 2. The "food" query returns a list of search results, these results have each have a link, DDG wants to display the favicon for each link. 3. To be clear, DDG does not store or log the IP address of the user doing the query. They do, however, know what was queried, so they know "somebody" somewhere searched for "food". They have to know this, they are a search engine after all. 4. Since DDG wants to show the favicon "privately", and they dont want to put that logic/work on the client side (which could leak your IP), so instead DDG finds the favicon internally. 5. A DDG server, completely separate from anything search-related is then tasked with finding the favicon for your "food" query results, lets say the #1 result is www.allrecipes.com, so a DDG server goes to www.allrecipes.com and finds the exact favicon location. 6. The "found" favicons are then stored in a cache, and displayed from the cache like this: https://external-content.duckduckgo.com/ip3/www.allrecipes.c... (and if no favicon is found in the local cache, you get a grey arrow by default) 7. I'd like to note, even with all this action, DDG doesn't know if you actually "visited" www.allrecipes.com, they simply know that some anonymous user did a search for "food", www.allrecipes.com was a search result, and a favicon was displayed. They dont know who searched for it because the users IP is not stored anywhere, they dont know if you visited www.allrecipes.com, they prevented you from leaking your IP to allrecipes.com since they didn't force the end user to load the favicon.

So whats the issue? What am I missing here?

PS: You know this works because after doing all these searches for food and seeing allrecipes.com (and even clicking allrecipes.com result in the DDG Mobile App or browser extension), guess what? allrecipes.com doesn't follow you around with re-targeting ads! Why? Because DDG prevented that from happening!

ddevault6• 6 years ago

This is not duckduckgo.com. This is the DuckDuckGo-branded end-user web browser.

tonyaiken• 6 years ago

The issue is with browser, not the search service.

glouwbug• 6 years ago

Looks like its time to ditch duck duck go

hota_mazi• 6 years ago

At DuckDuckGo, we do not collect or share personal information. That's our privacy policy in a nutshell

Except that you do, exactly in the way that the reporter of the issue explained to you.

But you choose to patronize them and ignore the issue.

Geee2• 6 years ago

I guess they collect statistics of the sites that people visit. This is anonymous but valuable information.

jacquesm7• 6 years ago

Your guesses are both wrong (see upthread) and irrelevant.

Geee2• 6 years ago

I just reread the thread and don't know what you're referencing.

foxhop• 6 years ago

DDG mobile apps ~= Web Browser or == Web Browser

I think that distinction needs to be made. I think DDG should treat this app as a web browser which means phoning home to this endpoint is unacceptable.

pochamago• 6 years ago

This doesn't seem like an issue to me.

zeckalpha• 6 years ago

Seems a bit much, but k-anonymity could work here. Hash the domain, take the prefix, get a batch of favicons back. They won’t know which you visited, but still get the benefits of consistent favicon support.

bad_user8• 6 years ago

And how would they get the "batch of favicons" in the first place?

jhasse• 6 years ago

When there's a miss, send the domain to the server so that it can fill the cache. You probably won't even have to do that as DDG could have already filled the cache with their search results.

smolder2• 6 years ago

That's an elegant fix, but for a problem that wouldn't exist if the feature was properly designed. (IOW, designed as a local function.)

jpangs88• 6 years ago

The favicons on the duckduckgo browser are often worse than other browsers in my opinion. For example the BBC website where DDG interestingly enough just uses /favicon.ico and the other browsers use the apple touch icon. (Information I found from just looking at the pages headers)

Don't really understand why they do extra work to get worse results... This feels to me slightly worse than just a privacy concern, it's a misunderstanding of their domain which leads me to the question of what else do they not fully understand.

The good news is that you can have the DDG search engine as a default in other browsers.

(I understand that the DDG browser is probably not their main focus and any lack of knowledge can potentially be just on their mobile browser.)

bad_user8• 6 years ago

Speaking of leaks, I never understood why people use DDG's bangs.

By using bangs you're sending your search history to DDG even when using search engines that aren't DDG.

Nextgrid3• 6 years ago

Because it's the easiest (and sometimes only) way to get this functionality on major browsers and browser developers have no plans to implement this functionality natively.

bad_user8• 6 years ago

Both Firefox and Chrome have custom search keywords:

https://support.mozilla.org/en-US/kb/how-search-from-address...

I maintain a set of such keywords in my Firefox, that I use on iOS and Android too.

The easiest would be to just use Google btw, as it does a reasonable job to give you the website you're thinking of just by mentioning it in the search query, e.g "Frozen imdb" (though it does a perfect job in this case with just "Frozen").

If you use DDG, because I'm assuming you value your privacy, sending your search history to DDG when you could avoid it doesn't make much sense.

tome• 6 years ago

you're sending your search history to DDG

I was astonished by this at first, but I think you must mean "you're sending all searches performed with bangs to DDG". I worried that you meant somehow the browser search history was being sent to DDG, but that seems impossible.

mkl• 6 years ago

I only use bangs when regular DDG search has failed me, and I'm quite happy to tell DDG of that failure.

pelliphant6• 6 years ago

but.. that is obvious?!?

If I don't want ddg to know what I google, ofc I won't do it by typing in !g in ddg...

I actually use bang functions because I want to help ddg out by informing them when I'm unhappy with the results I got from them.

bad_user8• 6 years ago

So ... you're expecting them to log your searches for later analysis, or at least meta data about your searches.

That's what they promise to not do and the whole point behind many people's decision to use them.

And it might be obvious for some, but it still makes no sense :-) given the browser is capable of doing it.

davidhyde• 6 years ago

Ubiquity did the same thing with their routers. They couldn’t understand why users had such a problem with their phone home feature that was on by default when the purpose of it was to ultimately “improve” the user experience. I didn’t buy their router as a result. I also removed kaspersky from my computer because I didn’t like their phone home feature. Turns out they were selling my data despite holding my trust as a security company. DDG, don’t turn this into a PR nightmare. We don’t trust anyone anymore. Privacy policies are worthless. Nobody cares about favicons anyway.

Source: https://www.theregister.com/2019/11/07/ubiquiti_networks_pho... https://palant.info/2019/08/19/kaspersky-in-the-middle-what-...

pelliphant6• 6 years ago

wow, didn't know that, thnx for the heads up.

Time to check what my ubiquity router is up to once I get home from work.

mhaberl2• 6 years ago

Product description (play store):

"Tired of being tracked online? We can help."

And then they track you.

Yes, that might not be intentional and is used "just" for the favicon, yes they might not use the info on the domains you visit for tracking you today, but the data is there.

Why not use that data tomorow "just" to see what kinds of pages their customers (browser users) are visiting so they can better place their ads.. and then maybe some other idea.. this is a path that many such companies went ("don't be evil").

You either respect the user privacy or you don't - there is no middle "just for this little feature" ground

mikaeluman• 6 years ago

I don't want to have to trust everything follows a policy.

It's much easier if I don't even have to trust you. Please change this.

throwaway_pdp092• 6 years ago

There's an interesting disease showing up here in the responses.

I accept DDG's statement that this is about a favicon and that they "do not collect or share any personal information", and despite that, I also agree with others that DDG should be on the safe side and just stop doing this small thing. It's just the safer and more moral thing to do (So DDG, as many are suggesting, plz stop doing it. Today is good).

But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do. There's no measure of proportion in the responses, someone is making a mistake then there's a wolfish, pack-like desire to get stuck in and hurt someone.

Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong. DDG isn't the big evil on the web but from reading some of these you'd think it was the 2nd google.

This isn't about DDG, just the proportionality of responses in public errors and what society you'd like to have.

(no affiliation to DDG)

warpspin8• 6 years ago

I think what angered people was actually that a company saying to hold privacy high was simply refusing to change something after a mistake was pointed out and instead kept on defending it with a technical argument, which makes no sense at all.

The reaction would have been actually a lot different if someone from the company admitted the mistake and promised it will be changed.

Update: Gabriel Weinberg has promised to change it, linking it here so it does not get buried in the pile of comments: https://news.ycombinator.com/item?id=23711597

asjw2• 6 years ago

Yeah people burnt witches in the past ...

If they are confident that this feature has no privacy implications they are right to defend the point, despite what people say or think

People don't run DDG, the company

People can use another search engine if they want

I'll keep using DDG anyway

koheripbal• 6 years ago

Read your comment again.

You are faulting someone for defending thier own argument. You suggest that people who do not cow and apologize to the mob deserve the anger and retribution the mob has to offer.

People have a right to think differently and express themselves without threats, bullying, or shaming.

The mob does not deserve apologies. The comment above is spot on - we've lost all sense of proportionality.

It is an indication of the modern online mob sickness that they always demand others beg for forgiveness.

What emotional void are mob participants trying to fill with the apologies of others?

SmokeyHamster• 6 years ago

defending it with a technical argument, which makes no sense at all.

Well hold on now. If there's a valid technical argument, and it's not a violation of privacy, why doesn't that make sense?

If people are so distrustful of DDG that they don't believe that argument, why use their browser under any circumstance?

hanspeter• 6 years ago

ignoring the genuinely huger issue of the amount of info and mining that google

By using DuckDuckGo you're doing the opposite of ignoring privacy issues in Google products.

And hence the reaction. Why use DDG at all, if they're not safely protecting your private data 100%?

TheUndead96• 6 years ago

This is my feeling exactly

asjw2• 6 years ago

Because some protection is better than no protection

Why use a condom which doesn't protect you 100%?

raziel2p• 6 years ago

You're okay with them having data on what you're searching for, but them potentially having data on the domains you're visiting from those search pages is where you draw the line?

I mean, I agree with the principle of storing as little data as possible but this isn't a huge deal in my eyes.

nsomaru• 6 years ago

Google don’t sell privacy.

Anyways, what’s this got to do with Google? “Privacy browser violates basic privacy to do something useless” is actually ridiculous. And the response is even more ridiculous! How does literally every single other browser do it?

I’m disappointed because I put my reputation on the line to recommend DDG to users based on...privacy. But here we see they actually do not hold true to their stated values. And they don’t even seem to care.

anyzen• 6 years ago

Google don’t sell privacy.

Of course they do. They sell users' privacy for advertising dollars.

I am not defending DDG here, as they are clearly in the wrong - but let's not pretend that their error is even close to what Google does.

jariel• 6 years ago

"Google don’t sell privacy."

What does DDG 'sell'?

Which is to say, how much have you paid them?

If you're not paying for it, then you are the product, that's the inevitable trade. There is no free lunch.

pelliphant6• 6 years ago

I disagree, or maybe we read different responses, but the ones I read where more critical of how (a) ddg (employee) handled this.

Didn't see anyone claim that this was on a google-level of bad, more like pointing out that google started out as a small company wanting to "do no evil", but slowly turned into what it is today.

Is it really that weird that people are worried that this might be the first of many small steps down the slippery slope?

afiori4• 6 years ago

Didn't see anyone claim that this was on a google-level of bad

The point is that people are reacting as if it was.

arkades• 6 years ago

I didn't have to go through the posted link for long to find people reacting as though this was more than a slippery slope issue:

But instead to react professionally and contritely you made it worse to stamp on the shards to make sure no useful piece of trust will survive.
I've just de-installed the Duckduckgo app and also won't be using their search engine anymore. Trust is lost. Their CEO can put his statement where the sun doesn't shine.

bartread2• 6 years ago

But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do. There's no measure of proportion in the responses, someone is making a mistake then there's a wolfish, pack-like desire to get stuck in and hurt someone.

Sadly people simply derive satisfaction from piling on like this. The pop psychology explanation is that it's because people are dissatisfied with their own lives and are lashing out at anything that allows them to vent that underlying frustration. It sounds plausible, but it also sounds like it might be an over-generalisation.

I think it's certainly fair to say - as depressing as it is - that it may be somewhat in our nature to behave in this way, that most or all of us may possess this characteristic to a greater or lesser extent, and that the current political, cultural, economic, and media climate is only serving to amplify that tendency.

baud147258• 6 years ago

Also with the current social media platforms, it's really easy to find and participate in such pile up.

classified4• 6 years ago

After we have been stung for the umpteenth time patience starts to wear thin.

slightwinder• 6 years ago

But... the reaction here is "they made a mistake, let's pile on like kids in a playground"

It's not one mistake, but several. Other then the initial mistake there is also the sloopy reaction and the fact they just closed the issue without bothering to fix it. And this was 1 year(!) ago. Nothing changed in the meanwhile. Now someone pushed it to public and after just some hours they reopen the issue and promise to fix it.

This is the reason why people react loud, because it works. And often it's even the only way that works.

afiori4• 6 years ago

Part of the point is that likely this has zero privacy implications (except potentially disclosing to someone monitoring your traffic that you are using their browser).

The mistake is rather an area of improvement where they can change something that respect privacy by policy to something that respect privacy by design.

the_cramer2• 6 years ago

Many argue that "other things are worse". This argument is invalid in my opinion. We're talking about this special case and if people feel like they need to openly show their opinion about it this is okay. Even if the response is big in numbers.

That just shows how much also the small things matter.

If you only care about "the biggest" or "the worst" you'll never get anywhere...

christophilus• 6 years ago

There's no measure of proportion in the responses

Man. You just described 2020.

Anyway, I’m now using the DDG browser, which until today, I didn’t know existed. I think DDG will do the right thing, ultimately.

jsnell2• 6 years ago

Oh, please. There's a million threads on HN complaining about Google and Facebook. Doesn't even matter if they did something, any post will still have those comments. You can't consider the proportionality of the response by just one thread.

It's really quite amazing that when a company that's hitched it's brand entirely to privacy first commits a big privacy faux pas, hides it for a year, and then doubles down on it not being a problem, you have somehow managed to turn the top voted thread to a discussion on the failings of other companies instead. Bravo.

phkahler• 6 years ago

It was not IMHO meant to deflect from DDG. It was a more general statement about peoples responses. I dislike trendy phrases but the one in play here is "outrage culture". Its not good and it's getting more common.

fooqux• 6 years ago

hides it for a year

Huh? It's an open source project. Maybe you consider them closing the issue on GH to be "hiding it" but I don't. And the plenty of people talking about it in that thread still apparently don't, either.

IMHO their response made sense. You're already trusting DDG with your search history which for most people might as well be a list of 99% of the domains they visit. If you trust their privacy policy for that, I don't see the big deal here.

I do agree that it should be changed, but only because as is it seems imperfect and I seek perfection in most things. Again, it's an OSS project. AFAIK, any one of these people comprising the screaming masses are free to fix it themselves. Personally, I don't think it's worth my time.

fierarul4• 6 years ago

You're right mr throwaway, let's be adults here and stop using DDG.

tux1968• 6 years ago

What would you suggest as a better alternative to protect your privacy?

Frost1x• 6 years ago

My guess is that DDG is being held to a higher standard because it's the liferaft competitor in a world of Google data harvesting, targeted manipulation and selling customer data.

Most complaining or being heavily critical about DDG are probably already upset to the point of abandonment with other services and they don't want the same trend to happen to this competitor (DDG). This sort of reaction is, IMHO, due to poor diversity of viable competitors.

In our societal structure, competitive options are the only things that keep power in check. I'm personally not entirely convinced you can have a reasonable amount of diverse competition in our economic system and there are some inherent equilibria that we tend to converge on over and over again in a market space (without corresponding massive social equalibria shifts).

If you do have any sort of faith left in our economic system, then you certainly want competitors like DDG to be different and be successful. Even if you don't have much faith, outside of say stringent regulation, supporting these sort of competitors is really the only practical option we have in the current state of affairs.

neya• 6 years ago

If you claim to fight for privacy and rise to popularity by shaming your competitors' evil anti-privacy (or shady) practices and do something exactly what the rest do, you deserve every bit of criticism. Why should you receive a free pass and the competitors shouldn't? Simply because of your marketing stance??

And for the record, collecting your browser history just to display a stupid favicon is the most ridiculous excuse I've heard in a long while. And I am not going to blindly believe them because they said that's what they use it for.

xvolter• 6 years ago

I agree with this.

As a privacy-first company, for them to make any decision that is for performance but adds additional privacy risk shows they make the wrong decisions.

They also left this for over a year after being told about it and part of the only reason it was caught was that their browser app is open-source. What about all their closed-source services, like their favicon service their browser apparently relies on.

We shouldn't blindly trust any company and a privacy-first company should be willing to assume we don't trust them and therefore it is up to them to prove everything they do.

I get a few of the responses on their Github page are now non-helpful, but the big point here is that by DuckDuckGo taking this series of actions (implementing the browser like this, ignoring a valid bug report about it, and only reopening the issue after massive push from users) it shows that DuckDuckGo isn't as privacy-focused as I thought, nor many others.

I've been using DuckDuckGo as my sole search provider for several years now, on all my devices, but I'm concerned about what else they may have done in the sake of "performance" over privacy.

OptionX• 6 years ago

"collecting your browser history just to display a stupid favicon is the most ridiculous excuse I've heard in a long while"

Though I agree the the implementation could be better, they should just check the head and the root for the icon and if not found that's that. But the possibility of something be used for malicious ends does not entail confirmation of it being used that way. Just because we have knifes in our kitchen does not make us automatically guilty of stabbing people. I find it easier to believe that this was actually, if misguided, an attempt to solve a problem rather than a nefarious plots to track users across the web.

dwild2• 6 years ago

But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do.

What about genocide too? Please people, stop with these "but the other bigger unrelated issue should get more visibility".

Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong.

Can you point me where DDG admitted they were wrong doing this? They didn't... they just explained why they did it but completely ignore the greater issue because they consider themselves "good". Just like that politician you may talk about, or Google, or whatever.

This is about DDG.

dwild2• 6 years ago

But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do.

What about genocide too? Please people, stop with these "but the other bigger unrelated issue should get more visibility".

Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong.

Can you point me where DDG admitted they were wrong doing this in their first response? They didn't... they just explained why they did it but completely ignore the greater issue because they consider themselves "good". Just like that politician you may talk about, or Google, or whatever. They are part of that bigger issue you mentions.

This is about DDG.

Luckily that pile of kids in a playground made them realize that mistake, they would have ignored otherwise (like they did on their first respond).

trashburger• 6 years ago

From st3fan's links, this[0] seems to be something that DDG developers can use. Took me about 30 seconds to dig it up from the repository.

[0] https://github.com/mozilla-mobile/android-components/tree/ma...

yegg• 6 years ago

Hi all, Founder and CEO of DuckDuckGo here. I’m literally just waking up and reading the comments here.

I’m new to this issue and happy to commit us to move to doing this locally in the browser and will have us move on that ASAP.

That said, I want to be clear that we did not and have not collected any personal information here. As other staff have referenced, our services are encrypted and throw away PII like IP addresses by design. However, I take the point that it is nevertheless safer to do it locally and so we will do that.

pelliphant6• 6 years ago

Thank you, this is the response people here want to hear.

And if this gets fixed in a reasonable timeframe, this is just one of those "everyone makes a mistake one in a while"-things, no big deal.

azalemeth• 6 years ago

If anything, it's much better than 'no big deal'. It's "We made this design decision, thought you would like it -- we've learnt, changed, and will avoid it later".

Can you imagine Google doing something similar? Heck, they're just about to throw the Android rooting community under a hardware-attestation DRM-filled bus.

NegativeLatency• 6 years ago

It's fixed: https://github.com/duckduckgo/Android/pull/878/files

jacquesm7• 6 years ago

Thank you Gabriel. This was what I expected.

chrispauley2• 6 years ago

This response validates my trust in DDG as has happened so many times before. Seriously cool company you've built here.

MfromB123456• 6 years ago

Really, just a response, saying everything is ok, validates your trust? Bought some wirecard stocks latly?

_hl_4• 6 years ago

I've already posted this somewhere else, but I'll copy it here again as well:

Maybe this should be an opt-in rather than an opt-out feature?

All in all its really not as big of an issue as people here make it out to be.

epi0Bauqu3• 6 years ago

Yes, after consulting with staff, I understand we thought it was more privacy protecting because we know our services are already encrypted and throw away PII, and so to get the favicon you could either (a) make another request to our known anonymous service or (b) make a request (or possibly multiple) to a non-anonymous service. On the other hand it is another request to a distinct domain that traverses another path on the Internet, albeit an encrypted one.

warpspin8• 6 years ago

This was not about search results, where their favicon webservice is in fact privacy increasing, but about the privacy browser and the favicons it displays, where it is privacy decreasing as it involved sending information about visited sites to a central authority while you are not on the DDG search engine. For example the TabRenderer will fetch the favicons from DDG instead of from the site you are actually visiting: https://github.com/duckduckgo/Android/blob/db728523240e37727...

Anyway, great decision by Gabriel.

classified4• 6 years ago

not as big of an issue as people here make it out to be.

Please refrain from speaking for others without being asked to.

bluesign4• 6 years ago

I think it would be more clear if you come up with a statement like:

‘we never used this data, other than showing favicons’

epi0Bauqu3• 6 years ago

We never used this data, other than showing favicons.

In fact, we didn't (and don't in general) collect any user-level data in the first place, per our strict privacy policy: https://duckduckgo.com/privacy

In this case, the way it works is you hit our favicon service and it returns the favicon, not using any PII in the process, and our web servers are configured not to log any PII. In other words, our system is technically designed and we are legally bound by our privacy policy to not use this data for anything other than showing favicons.

Terretta2• 6 years ago

To be more clear, your staff, and you, have said PII ‘like IP addresses’, and have said ‘thrown away’ some places and ‘not collected’ others.

Contrary to this framing, it’s not possible to not incidentally become aware of every single browser users’ usage timing and user IP addresses if the browsers are phoning home this way — a colloquial understanding of ‘collect’, not the James Clapper NSA dodge definition of ‘collect’. Most normals think of collect as become known not as permanently store. You knowing it means others can know it if you break trust or are required to comply with authorities.

And regardless of end-to-end encryption, that this user is phoning home to your fave icon endpoint, when, and from what IP, is revealed to every ISP in the chain. You’re leaking browser usage telemetry to every single party to that traffic — the source IP address PII you mention is in unencrypted metadata.

The fact this browser connects to that endpoint reveals demographics (choice of privacy browser) and behaviors (when and how much web surfing) to e.g. ISP or nation state firewall operators who are certainly not bound by your ‘just trust us’ privacy policy.

Privacy policies are a patch for insufficient privacy engineering.

To be a strong privacy browser you could consider what it would take to be “NSL proof” such that if handed a national security letter with gag order, you cannot comply. That is not the case with this faveicon telemetry endpoint.

0xfate• 6 years ago

Just to be fair, as a matter of fact, you surfing that site is revealing you, surfing that site to your ISP and state actors, in the first place. A change, where to get the icon from (origin vs ddg), will not change this fact. It is all about ddg not getting to know, which sites you are surfing, when not searching for it on ddg. Which should, indeed, be a no-brainer.

m463• 6 years ago

Thank you, keep up the good work!

(I always knew there was a business model for privacy and I'm glad someone is working to figure it out)

jsnell2• 6 years ago

Thanks. What kind of process changes are you putting in place to prevent things like this from happening again?

worble3• 6 years ago

It's a little confusing to see 2 accounts, with very different usernames (epi0Bauqu and yegg), that appear to be posting as Gabriel Weinberg . Are these both legitimate? And if they are, what's the reason for two of them?

epi0Bauqu3• 6 years ago

Yeah that's my bad. epi0Bauqu is my original account, but since no one knew who I was under that account I, some years later, made the yegg account and I try to post from there. The issue here is I logged in to post the comment, and then switched to my laptop where I was already logged in this other account.

fierarul4• 6 years ago

That said, I want to be clear that we did not and have not collected any personal information here.

So, you do collect information just not the kind you would classify as 'personal information'. I wonder if my personal domain with my full name qualifies?

There is no way this feature would be created in a company built on privacy considerations.

Like, wow!

orisho• 6 years ago

Sure it could. A junior programmer aware of the values but not the repercussions could easily overlook this without any malice or disregard for privacy.

worble3• 6 years ago

So, you do collect information just not the kind you would classify as 'personal information'. I wonder if my personal domain with my full name qualifies?

Considering they have operations in the EU, I would imagine that would fall under what the GDPR considers personal information (https://gdpr.eu/eu-gdpr-personal-data/), or risk being in breach of it.

mv4• 6 years ago

Good. Thank you.

ddevault6• 6 years ago

Thank you for re-opening and prioritizing this.

However, this problem demonstrates gross incompotence for a browser team supposedly concerned with privacy. Will you please do a post-mortem on how this code made it through your code review process in the first place, as well as how it managed to stay in place for a full year after it was pointed out that it represented a privacy problem?

"Sends every URL you visit to the vendor's servers" is the single worst thing DuckDuckGo could have done for privacy in this web browser, and that needs to be accounted for. There was a major failure in the code review process, ticket review process, and in how you treat your community. A standard marketroid "by design" response with washy promises that "we'll take very good care of this highly sensitive personal data, just trust us" is not something I want to see in the future from this team.

[reposted from GitHub]

elliekelly• 6 years ago

I’ve worked with many companies who have demonstrated “gross incompetence” when it comes to privacy and information security. This is absolutely not an example of gross incompetence.

I agree that for a company built around privacy even the appearance of impropriety needs to be avoided. DDG holds themselves to a higher standard and their users hold them to a higher standard.

This was a design flaw and a process flaw. DDG prioritized speed and efficiency over privacy (or in this case, perceived privacy) and I suspect there isn’t a soul on HN who hasn’t made that trade off at some point. They assessed the cost/benefit and risk/reward and it turned out their assessment was wrong. Now they’re fixing it. It happens. But to call this gross incompetence is really blowing it completely out of proportion.

akerl_• 6 years ago

You can’t think of anything they could have done that would be worse than sending URLs to their lookup server? It’s the single worst thing?

My browser syncs URL history between my devices, and that’s a feature that I value about it. Your comments on this topic seem to suggest that all users are making the same decisions about what is acceptable usage of their data, and that’s pretty obviously not true.

jiofih3• 6 years ago

Maybe you’re taking this a bit too far? They explicitly state they will not store your data anywhere, and the main safeguard you have for that is your trust in them, not this one specific line of code somebody happened to notice which can’t even break that promise on its own.

polycaster• 6 years ago

If DDG cannot fetch the favicon in different, reasonable way, then the question is whether or not the ability to display a favicon in search results is really worth it.

Personally, no.

gowthamgts12• 6 years ago

created a new issue to track this again: https://github.com/duckduckgo/Android/issues/876

renewiltord• 6 years ago

Haha, amazing to witness. This is the problem with catering to this crowd: your audience is suddenly full of people who just want to see you fail. Good luck, DDG.

mhaberl2• 6 years ago

I don't agree.

I really like DDG - it works good, it is fast and it does not use my personal search for giving me "better ads" or "better search results" that put me in a filter bubble.

But there is a different issue here at play; because of errors like this the whole DDG brand gets a bad rap - and thats not only bad because of the risk of people losing a google alternative but because it is real easy to exploit situations like this for google-like companies to give an impression that "all this privacy thing is bs, all companies work in a same way". There are a lot of people that are not really sure is this "privacy thing" is worth the inconvinience of swiching to some other search engine/browser/app and situations like this one are not helpful in that regard.

Lot of folks are aware of this and are displeased for risking brand confidence of such a visible privacy-concerned company for miniscule gains like performance gains for fetching a favicon for the first time - just fetch the favicon after you display the rest of the page and cache it, maybe dont even try to fech it if the connection is poor - who cares really

classified4• 6 years ago

By now it has been sufficiently proven that it is physically impossible to even exist without sending surveillance data to someone on the internet. We should probably update the laws of thermodynamics to include that.

olafure• 6 years ago

I think we're due a full disclosure on this favicon service, what information is collected and what is stored.

DDG has repeatedly said that they have "not collected any personal information".

For example,

1. Does the service store the fact that it got a request for a domain?

2. Does it store any ID along with that information and if so, how unique is that ID? How is it generated and what is it linked to?

3. What other information is stored along with the request?

4. How does DDG process this information?

5. Who has or can get access to this information?

eightlimbed• 6 years ago

Can someone please explain like I'm five how this line of code sends the domain a user visited to DDG's servers?

oplav• 6 years ago

Every time you visit a website on the Android version of the browser, instead of requesting the favicon from https://example.com/favicon.ico, the app is calling out to https://icons.duckduckgo.com/ip3/example.com/favicon.ico.

Since DDG owns the icons.duckduckgo.com service and the domain you were interested in is in the request to icons.duckduckgo.com, you've sent the domain to DDG's servers.

sonicggg• 6 years ago

Something is not adding up. Why would you go through so much trouble and over-engineer a favicon retrieval service? Really, favicon? Since when did they become so essential?

I'm pretty sure 90% of websites provide one in a standard way. If not, just draw a letter there, or anything.

But I don't know. I think that either there is more to this story, or DDG team completely lost common sense.

WClayFerguson2• 6 years ago

It is a hilarious excuse for DDG to claim they are doing this for a favicon. Even if DDG is legitimately not using the data they are definitely collecting the data.

The problem with that is that it requires users to "trust" DDG, which is not how the world works today. If you are a company that collects info, and you expect users to trust that the info will remain safe, secure, and never get misuse that is downright foolish for anyone to believe a word of that.

We all know that DDG cannot claim it's impossible for them to get hacked and have all that data leak out. Hacks happen all the time and so the solution for DDG is to simply NOT collect the data, rather than collect and claim it's all secure.

And we all also know DDG has (or will) get a NSL (National Security Letter) from the NSA to secretly turn over the data anyway, and when that does happen the DDG employees are not even allowed to admit it ever happened.

twirlock• 6 years ago

I didn't know they had a browser. I'll have to give it a try. Can't be any worse than Google's browser. Or their OS. Or their video monopoly. Or their search monopoly. Or their secret partnerships with governments. Or their ad monopoly. Or their email monopoly.

Angeo34• 6 years ago

Whoever ever put their trust into American for profit companies which use slogans like "private secure and fast" should not be surprised at leaking all their data.

I never got how people trust companies like ddg or Brave. If you don't trust Google and Apple why would you trust a smaller company in the same jurisdiction. They will be forced to hand out all data as well regardless what they say.

DuckDuckGo browser seemingly sends domains a user visits to DDG servers

Related Stories