marcus westbury

my life. on the internets.

marcus westbury header image 2

How the Internet filtering trial works and fails (via my brother)

December 16th, 2009 by marcus


I’ve been tyring to get my head around the results of the Federal Government’s internet filtering trial. Rather than undertake hours of laborious research off my own bat i decided to consult the resource that all people i know turn to when stuck with a technical question: my brother Stuart.

His assesment:

Here is a bit of background. The Enex report (which is fucking big) on the live trial explains that there were 3 technologies used by the participating ISP’s. Some in combination with each other.

Proxy server

The simplest solution. Just blocks web traffic based on a blacklist using a standard proxy (caching) server. This would work in small scenarios but is extremely simple to bypass unless they block other network traffic. They have not said that the proxy trial participant did this. I do this at work to block known malware sites based on a blacklist. In larger implementations this is just not feasible due to the massive amounts of data you would have to pass over these servers. It is not scalable and not suitable for ISPs the size of Telstra or Optus.

Pass by filtering

Not all traffic is inspected. This would need to be used for much larger implementations. Instead of funneling everything via the proxy, they maintain a list of IP addresses that are blocked using border gateway protocol (i wont go into that) on border routers. If you are attempting to access an IP address that is on the blacklist, your traffic is then funneled through a proxy server to filter it for the actual URL. You cannot just block an IP address for undesirable content. This is due to shared hosting environments that often have the same IP. Blocking the undesirable IP would potentially result in blocking harmless stuff on the same web host. This also just relies on a blacklist anyway. Again, there is no mention of blocking ports or protocols like bittorrent, P2P technologies, IM, VPN or anything else that could be used to transmit smut.

Pass through filtering

Pass through filtering is the scariest one. It performs DPI (Deep Packet Inspection). This one can identify undesirable content inside individual packets of data but it is also by far the most resource intensive to implement. It has the potential to inspect torrents, IM etc but will still be defeated by encrypted technologies live VPNs. Without the ability to decrypt, then inspect a VPN packet (making the “Private” in virtual private network redundant), the only way to stop it accessing nasty content is to block them all. The economic implications of this are huge. They just won’t do it.

Some other stuff

Of 37 circumvention tests performed against the filters. The successful block rate ranged from 8.1% (proxy / pass by) to a much higher 94.5% in the case of hybrid proxy / DPI methods. You can be assured that a) this method will not be implemented without the government subsidising banks of super computers and b) the circumvention that worked against it is the holy grail of defeating this thing. VPNs. It will also be capable of serious false positives.


I looked mostly into Participant ISP #5’s results as it had the most success in blocking circumvention attempts – 94.5%. It also had by far the worst results in terms of performance degradation. I didn’t do any number crunching but the graphs show at least 50% in a lot of cases. This will not be implemented as the final solution. It would be insanity.

It’s also important to note that these tests are also ludicrously based on people getting access speeds of 8mbit (FTTN specifications) / sec in a trial that involved very small numbers of real clients. What happens when the NBN rolls out and is supposed to supply most of the nation with 100mbit connections? This whole thing will need massive reassessment.

Coming tomorrow: my brother on how your fourteen year old will be able to bypass the filter anyway.

Similar Posts:

Tags:   · · · · · · 5 Comments

  • Delicious
  • Facebook
  • Digg
  • Reddit
  • StumbleUpon
  • Twitter

Leave A Comment

5 responses so far ↓

  • 1 Kevin Rennie Dec 16, 2009 at 4:48 pm

    The nature of the restricted Content is a greater worry tahn the technology. My analysis is at the link.

  • 2 john Dec 17, 2009 at 8:39 am

    Question for your brother, by how much will this slow up the web? And how dos it all compare to say chinas web censorship?

  • 3 Stuart Westbury Dec 17, 2009 at 9:45 am

    Hi John,

    It’s really hard to say how much it will slow things down because the report is incomplete. I personally get 15 megabits at my house and these trials used a maximum of 8. The NBN will apparently aim to supply 100! It’s very hard to tell with such a flawed trail.

    Another thing to take into account was how small the uptake was. One of the participating ISP’s was Nelson Bay Online who are hardly a giant in the industry. They had a whopping 15 users opt in to the trial. I could also filter 15 users at 8 megabits and my graphs would look fantastic. This is a question of scale and when we are talking about something national, there will undoubtedly be issues.

    Some filtering tech – like pass by filtering – will possibly have a minimal impact but will also be the easiest to bypass. In my opinion, the more successful systems like DPI are just not possible on such a scale.

    I don’t believe this will be anywhere near as bad as the great firewall of China. The Chinese do a lot more. They hijack DNS servers. They claim servers that exist externally on the internet.
    This technique is quite effective but is a much heavier approach. They also block complete subnets of internet address space as well as the type of filtering the Rudd government wants to implement.

    The scariest thing about the Chinese system is that they have companies like Yahoo and Google on side. These companies work with the Chinese government to create versions of these search engines that filter undesirable keywords.

    As pretty much any Chinese activist or teenager will tell you though, it’s still pretty easy to get out.

  • 4 john Dec 18, 2009 at 9:22 am

    so, will it all : fall in a heap,
    Or be a pointless & expensive(but harmless) sop to moral panic
    What dos your brother think

  • 5 john Feb 7, 2010 at 3:51 pm

    wondering -have you had a chance to ask your brother, as to what he thinks the’ actual’ scheme is meant for; is it just a piece of posture, aimed at the mug punters or what?