TorSearch launches to be the Google of the hidden internet

http://venturebeat.com/2013/10/10/torsearch-launches-to-be-the-google-of-the-hidden-internet/

by IceyEC • 13 years ago

94 48 13 years ago

You know, I'm somewhat suprised Google isn't actually crawling tor. Or maybe they are, who knows?

IceyEC16• 13 years ago

There are a few cases I've seen of Google actually picking up onion.to links but they tend to rank terribly, maybe for their name

GunlogAlm2• 13 years ago

Yeah, there are lots of onion.to sites indexed. Maybe the ranking is to do with the fact they experience so much downtime?

vezzy-fnord• 13 years ago

I recall the engine once actually picked up a genuine .onion domain (it was the Hidden Wiki, IIRC).

It's safe to assume they do crawl the Tor space.

hnha2• 13 years ago

onion.to is not "crawling tor sites". It's crawling someone's public web proxy's copies.

vabmit• 13 years ago

.onion crawling was a 20% time project in 2009/2010 (IIRC). I'm sure you could find the announcement if you searched for it. Google does crawl and index.

neur0mancer• 13 years ago

https://encrypted.google.com/search?hl=en&q=site%3A.onion

No results here. If they are crawling, the results are not shown.

kriro• 13 years ago

Interesting. They should monitor non-Tor access to the site for a bit for kicks. I have a feeling that would provide some interesting results :)

IceyEC16• 13 years ago

Unfortunately, traffic coming from Tor and traffic coming from the onion.to forwarder all looks the same to me :(

gizmo6862• 13 years ago

Could you fingerprint the browser to determine how many users are using the Tor Browser? Obviously this will miss the people who just use a normal browser through Tor. Or ask onion.to to log how many requests they are forwarding to you. Although, this will miss any other proxying service that happens to be running.

dmix• 13 years ago

To clarify, TorSearch isn't accessible from clearnet (from my understanding) so he can't monitor->correlate individuals activity. Such as a nefarious search.

aaron42net• 13 years ago

If you are interested in tracking from within Tor vs in via tor2web, look for the header "X-tor2web: encrypted" on HTTP requests.

cmaury3• 13 years ago

There are already multiple search engines for Tor, including DuckDuckGo.

The problem with Hidden services/ TOR isn't search. It's the fact there aren't enough legitimate/trusted websites that appeal to users outside the hardcore privacy/security crowd.

edit: fixed Tor capitalization.

IceyEC16• 13 years ago

DuckDuckGo isn't a search engine for Tor, it just provides a hidden service interface to its regular search engine. If you want to make a hidden service to help improve the overall quality, let me know and I can help you get it all setup.

cmaury3• 13 years ago

I thought you could limit searches to .onion domains. Oh, well. Thanks for the offer to help. There are a few ideas for site that I would want to see on the Tor network, but I don't think they would have the draw enough traffic to get people running TOR on the regular.

Though writing this gave me an idea: is there a .onion Torrent index?

edit: fixed Tor capitalization

GunlogAlm2• 13 years ago

DDG doesn't search Tor, it's just a .onion version of their regular site.

Also, it's Tor not TOR. https://www.torproject.org/docs/faq.html.en#WhyCalledTor

gesman• 13 years ago

Your friendly search solution to find the stuff you momma (and uncle Sam) don't want you to see.

knowaveragejoe• 13 years ago

I notice the Telecomix logos in the image. Did they have anything to do with it? No mention of them on the page.

IceyEC16• 13 years ago

They didn't, I'm surprised the reporter used their logo for it

proexploit• 13 years ago

It looks like he used this image (http://www.flickr.com/photos/xp0s3/7851153390/in/photostream...), potentially just searching on Flickr for something eye catching and due to the "We're watching you" concept?

p4bl03• 13 years ago

It doesn't have a lot of .onion websites indexed. The two I tried to find are not in their database (for one of them I got a result though, because it is mentioned in the hidden wiki).

I don't see any way of submit a site for crawling.

IceyEC16• 13 years ago

https://kbhpodhnfxl3clb4.onion.to/add-domain

p4bl03• 13 years ago

Thanks.

By the way, I can't get the https version to work when not using .onion.to (it's not important then, but still I prefer to report this to you).

cpncrunch• 13 years ago

That will be useful for the NSA and FBI.

pbharrin• 13 years ago

Awesome! The first thing I saw in autocomplete was my book. I assume people are searching for discussion groups, not bootleg copies.

IceyEC16• 13 years ago

It doesn't have autocomplete... I bet your browser autocompleted on the field named q

hnha2• 13 years ago

nice potential information leak, is such autofilled text available to the website via javascript?

brokenparser• 13 years ago

How long until the NSA/GCHQ backdoors this site, if they haven't already? Would the thought police jump to conclusions if one uses this to search for classified documents? FBI files are probably of no use to me, so I'm not interested in those. But I hate it if people (in particular journalists) are considered to be some kind of al-shabab just for exercising democracy (i.e. holding their governments accountable under the exact letter as well as spirit of applicable legislation).

IceyEC16• 13 years ago

the reason I decided to implement this as a hidden service and not expose it to the regular internet (except through things like onion.to) is that my server sees every request as coming from localhost; all requests are processed through Tor so no matter what you search, as long as you aren't coming in through a Tor mirror, nobody can tell who you are

w-ll2• 13 years ago

Thats not true, its specifically the attacks where they have many nodes, and if they can get on the entry and exit, the've identified that person.

Mizza• 13 years ago

Easier to enumerate Tor hidden services than to spider.. http://freehaven.net/anonbib/cache/oakland2013-trawling.pdf

znowi• 13 years ago

Not sure if they promptly rebranded, but the search is called Torch actually. Here's a screen of an example search [1]. Note the query time - it's quite slow.

[1] http://i.imgur.com/4tx4nzt.jpg

Quick guide to access Torch on the Tor network:

1. Install Tor Browser (customized Firefox) https://www.torproject.org/projects/torbrowser.html.en

2. Extract and run "Start Tor Browser"

3. Go to Torch at http://xmh57jrzrnw6insl.onion

IceyEC16• 13 years ago

Torch is an entirely different search engine at a different domain; I run TorSearch and built it from scratch partially because of how terrible my experience using Torch was.

a1a• 13 years ago

The problem I see is that you can't really trust a search engine providing links to hidden services. Since hidden services doesn't really use "understandable" domain-names it's very easy to duplicate a website.

What is to say the owner(s) of the search engine isn't targeting journalist and the link to the newspaper I found is a dupe of the 'real' site?

This is by the way a problem in general with hidden services.

EricBurnett• 13 years ago

The larger internet has the same problem - duplicate content farms, making the "original" of a piece of content hard to discern. It's certainly not a solved problem but by tracking which copy is seen first, using links to imply trust, etc, a reasonable search can still be provided. You'll never have 100% accurate attribution, but that's kinda the point. And when it comes to trusting the search provider, well, how do you choose now beyond seeing consistent result quality?

Agreed trust is a problem with hidden services in general, but I think it's one we'll solve by reframing what it means to 'trust' a site in the first place.

TorSearch launches to be the Google of the hidden internet

Related Stories