China has reportedly developed a system called “night crawlerâ€? that can “locate and blockâ€? websites located in China (by IP address range) that have not registered with the Ministry of Information Industry (MII). Nart Villenueve of Citizenlab explains what this system really is about. Of course we already knew for a long time that they have the great Chinese firewall and that they shutdown internet cafes, weblogs, and fora that are not in lign with current Chinese policies. For more information on internet in China read this in depth article: “China: The Net Effect” by Steven Cherry.

But not only communist and dictatorial countries use filtering software. More and more companies and countries use software like smartfilter to deny people from accessing certain sites. See smartfilters promotional flash movie, which explains how it works. And yes, also European companies use netfilters more and more (dutch article about netfilters in companies).

As a result of reading articles like this wired article on internet censorship, Richard Rogers and Auke Touwslager are mapping routes to avoid internet censorship as well as mapping the countries which have most censorship. I’m interested in their findings and will give a link to their research later on.

To be able to monitor which sites are blocked by which countries, I wrote a script that first got a list of public proxy servers, and then queried a whole bunch of sites through those proxies (see wikipedia on proxies if you don’t know what they are). The problem is that I don’t know which rules are enforced on those proxies. Each proxy can specify which sites it blocks and I don’t know if the proxies I found are the government controlled proxies. Therefore I cannot do good empirical research. I guess citizenlab’s current approach is the only way to go: put your own proxyservers in all countries through which you query sites. This way you know you will be routed through the countries’ proxy servers and not through a proxy with different rules. Bummer for my research

For more information on internet censorship see http://cyber.law.harvard.edu/filtering/, the open net intiative, and wikipedia on internet censorship.

Update:

Users of Microsoft’s new China-based Internet portal have been blocked from using the words “democracy”, “freedom” and “human rights” in an apparent move by the US software giant to appease Beijing. … Microsoft is not the only international tech company to comply with China’s stringent Internet rules. Yahoo! and Google — the two most popular Internet search engines — have already been criticized for cooperating with the Chinese government to censor the Internet. (link)