Open Rights Group has launched a UK Censorship Monitoring API and Botnet

https://www.blocked.org.uk/

by namos • 12 years ago

82 34 12 years ago

blocked.org.uk

namos6• 12 years ago

For the past 8 months we've been working on building a series of probes;

- Android https://play.google.com/store/apps/details?id=uk.org.blocked...

- Python https://github.com/openrightsgroup/OrgProbe

- Raspberry PI images https://github.com/openrightsgroup/lepidopter)

And an open API to power them: https://wiki.openrightsgroup.org/wiki/Censorship_Monitoring_...

This is helping to build a picture of what the UK Governments Internet Blocking/Filtering/Censorship actually looks like.

You can submit URLs to be tested by visiting https://www.Blocked.org.uk by installing the Android app or by tweeting a URL with the hashtag #IsBlocked

Once we receive a URL it is dispatched to all probes over a period of time where it is checked against a know list of blocking methods.

The results are returned back to the API for public evaluation.

For example: https://www.blocked.org.uk/results?url=https%3A%2F%2Fnews.yc...

Please help us build a picture of UK Internet Censorship by adding URLs to be tested, by helping develop new probes (browser extensions, an iPhone client etc etc) or just by spreading the word that https://Blocked.org.uk is available.

Please also see https://www.blocked.org.uk/faq which will explain the level of filtering enabled on some of the probes which will explain why some sites may appear blocked even if you, being on the same ISP, can reach said URL.

vidarh3• 12 years ago

It would seem like some classification of the blocked URLs plus a suitable extension to use willing proxies would be quite useful as an exercise in making the blocklist completely pointless.

I'd imagine a lot of people that have issues with e.g. running a Tor node would be perfectly happy to offer up proxies offering up various subsets of the blocked sites whitelisted by category.

mike-cardwell• 12 years ago

Please consider freeing the data and make it downloadable as a simple CSV. I'd like a list of all of the sites blocked by particular ISPs...

namos6• 12 years ago

Snapshots are made available: https://www.blocked.org.uk/assets/data/alexa_100k_isp_latest...

gone35• 12 years ago

Whoaa why the hell is The Brights' Net being blocked by BT??!? Since when is atheism considered "censorable" content in the UK??

This is madness.

https://www.blocked.org.uk/results?url=http://www.the-bright...

finnn• 12 years ago

Have you considered setting a user agent other than Python Requests default? Probably not necessary, but good practice. Also, ironically, the app is not available on any of my devices.

justincormack2• 12 years ago

Ah I knew something was in the works as Andrews & Arnold were writing about obtaining connections from ISPs to test what was blocked. Great to know what is being censored (if you are not on A&A).

dfkvldfmv2• 12 years ago

"(if you are not on A&A)."

Or if you're on any of the ISPs who do not have filters, including but not limited to A&A.

SideburnsOfDoom2• 12 years ago

Who are the other ISPs who do not have filters?

spingsprong• 12 years ago

I've been checking various websites I go on that are completely fine for kids. It's amazing what's getting blocked. One example according to that website is, TalkTalk blocks "http://www.writingexcuses.com/" It's a podcast that teaches story writing.

Blocking that is insane!

And the blocked website itself is blocked on two ISPs https://www.blocked.org.uk/results?url=http%3A%2F%2Fblocked....

TazeTSchnitzel• 12 years ago

I think TalkTalk is either using a whitelist or an oversensitive keyword filter.

namos6• 12 years ago

The TalkTalk filter is the default "KidSafe" level

vidarh3• 12 years ago

My personal website is blocked by Talk Talk apparently. I guess compiler writing must be harmful for children.

justincormack2• 12 years ago

If you test www.blocked.org.uk it doesn't show as blocked so either it changed between scans or there is an issue here. Maybe someone can test.

namos6• 12 years ago

www.blocked.org.uk vs blocked.org.uk AFAIK

blueskin_4• 12 years ago

This is with the new government mandated opt-out censorship enabled (except AAISP, who don't offer it) - if that is disabled, most blocked sites would be accessible, except that sites with 'adult content' are still blocked by default on mobile networks (EE, O2, Three), which is an account setting with age verification to enable.

dfkvldfmv2• 12 years ago

Except every ISP that isn't a mobile operator or a major UK wireline ISP (e.g. BT/TalkTalk/Sky/Virgin). Including Andrews and Arnold, but also including many smaller ISPs like Zen or IDnet or Xilo and others.

A&A have played a massive blinder on the media front, by saying that they aren't doing something that no one is legally requiring them to do (it would be interesting to see what they do if they were actually forced to, though). It's also worth pointing out that the mobile networks have had filters for years, because of Labour government pressure and not because of the "wall of Cameron" or whatever people are calling it.

I also like how the site says "checked on the main UK ISPs" and then includes A&A, which no one would ever say is a "main UK ISP". They're the nichest of the niche. They're an extremely good ISP (although you can get similar quality from others at a fraction of the price), but they're not a "main ISP".

I would guess that A&A have paid towards the ORG or are contributing something and that's why they're getting prominence.

blueskin_4• 12 years ago

I just learned that one of my personal sites is blocked on BT and TalkTalk... there is zero adult or other inappropriate content on it. Time to complain, I guess, but I'm sure they will just go straight to /dev/null :(

dspillett2• 12 years ago

Make sure you update the site to state that it is currently blocked by these ISPs, you don't know why, you have asked them to investigate, but you have not heard nothing back and until they respond it is completely out of your control.

That way anyone who can access it from work but not home (for example) or recommends the site to a friend who finds it doesn't work, will have a clue as to what the problem is (and it isn't your fault).

If your site is of significant general interest then this may also increase the chance of the ISPs acting quickly.

AlyssaRowan2• 12 years ago

Have you considered legal action?

blueskin_4• 12 years ago

Nope. It's a medium sized site I don't get any money out of, not linked to any business. I suppose I could theoretically small claims court them, but I'm not sure if I'd win as they can probably piss about with their customers' connections in whatever way they please due to their T&Cs :(

lucb1e2• 12 years ago

Wow, I had not expected this. My personal blog (which is about technology) is being blocked by one of the ISPs. I don't host porn nor have I ever, so why is it blocked behind their porn filter? Interesting....

https://www.blocked.org.uk/results?url=http://lucb1e.com

Edit: TalkTalk live chat support reports that I am not being blocked. Citation:

I can confirm that is not blocked by our Adult Filters as your website does not have any adult content.

So it seems blocked.org.uk has a false-positive...

blueskin_4• 12 years ago

TalkTalk are a completely awful ISP. Not sure what the US equivalent would be, but they're the kind who do promotions of "Broadband for £3/month"-level. You get what you pay for service-wise, but they are also by far the most enthusiastic ISP around when it comes to censorship; they have a spider that follows their users to all URLs they visit to categorise the sites[1] (giving them the nickname StalkStalk), and have been doing this long before the current set of idiots in Parliament forced censorship in (at least since 2011).

I'm in a similar situation, one site I run is blocked both by StalkStalk and BT. No porn, no other 'adult' content; it's deliberately kept PG-13 to use the US ratings.

[1]Check your logs for HuaweiSymantecSpider. It obeys robots.txt though, so a quick "Disallow: /" in robots.txt stops it, although they still grab content from unencrypted pages via DPI.

lucb1e2• 12 years ago

No HuaweiSymantecSpider in my logs in the past 8 months (grepped access.log), but the site is available over both http and https so they might have classified it using DPI (talk about evil...).

namos6• 12 years ago

TalkTalk has more than one filter setting.

Ask them if your site is blocked at the "Kid Safe" level (the default) which is what the probes run on.

endersshadow• 12 years ago

BT is blocking my site about my homebrewed beer. That's fun.

SideburnsOfDoom2• 12 years ago

My geek blog and a couple of other similar ones are also blocked by TalkTalk. It seems to be a bulk problem. I have emailed them asking for an explanation.

One talk-talk subscriber has reported that he can see these blogs just fine. Though he does not have the "kid safe" setting turned on, of course. "Blocked" in this context probably means "blocked when using the most restrictive settings".

jamesbrownuhh• 12 years ago

Sad to see that The Best Page In The Universe (http://maddox.xmission.com) seems to be blocked by a non-zero number of larger ISPs.

Obviously all those children need to be protected from harsh reviews of their artwork.

Open Rights Group has launched a UK Censorship Monitoring API and Botnet

Related Stories