The Guardian Is Being Swamped with 'Dark Traffic'

Re:The Guardian Is Being Swamped with 'Dark Traffic' by randunel on Tuesday November 04, 2014 @05:02AM (#8555791)

I built my personal extension which blocks all the traffic from Dan Pollock's list [1], then blocks all the traffic from major service providers (google analytics, etc) and social networks (fb, tw, google, etc) when not on their website. Referral and user agent headers removed, I haven't found the need to remove other headers. Currently working on preventing (and manually allowing) all xhr/script/image requests 2 seconds after the main frame has loaded.

The internet is a lot faster for me, and the battery appears to last longer. So far, I've only had problems logging into instagram, but once the cookie is set, I can re-enable blocking ads and tracking.

I guess I am one of those 'dark traffickers' :P

[1] http://someonewhocares.org/hosts/

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by cantlin on Tuesday November 04, 2014 @05:27AM (#8555872)

Looks like that list hasn't been updated since our domain switch. If you really want to block our internal analytics (which are in practice fairly harmless) replace "hits.guardian.co.uk" with "hits.theguardian.com".

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by jacquesm on Tuesday November 04, 2014 @06:22AM (#8556009)

Nominated for the most classy comment of the month. That the product manager of The Guardian would take the time out from his (no doubt) busy day to help a user to block tracking is an amazing display of trust in that the user knows best what is good for them. Thank you.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by pmoriarty on Tuesday November 04, 2014 @06:29AM (#8556031)

It's more like he knows that helping one single user like this isn't going to make a dent in the tracking they do on virtually all other users.

Now, if they voluntarily stopped tracking all or a significant portion of their users, I would be shocked.

Of course, that isn't going to happen.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by jacquesm on Tuesday November 04, 2014 @08:21AM (#8556335)

Of course there is always a way to put a negative slant on just about anything.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by joosters on Tuesday November 04, 2014 @07:30AM (#8556199)

I wonder why companies who host their own analytics choose to use a separate request to track users?

Surely it is better just to get the data from your own content web server logs ? Wouldn't you get the same information while saving on the extra HTTP request? It would make the site loading times slightly faster as well.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by strommen on Tuesday November 04, 2014 @08:52AM (#8556454)

Server logs will track all requests from crawlers, accelerators, and cancelled navigations. An AJAX callback will only happen for real human visitors that actually view the page.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by atmosx on Tuesday November 04, 2014 @06:36AM (#8556047)

I admire the fact that you offer this kind of info :-) thanks mate, always admired the Guardian for his articles, authors & general stance towards privacy.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by aw3c2 on Tuesday November 04, 2014 @07:30AM (#8556197)

Considering them telling almost 20 other domains about your visit, I would not call that stance on privacy a strong one: 2o7.net, ajax.googleapis.com, chartbeat.com, chartbeat.net, dqwufkbc3sdtr.cloudfront.net, facebook-web-clients.appspot.com, google.com, googleadservices.com, googletagservices.com, imrworldwide.com, mathtag.com, ophan.co.uk, optimizely.com, outbrain.com, scorecardresearch.com, twitter.com, wunderloop.net, www.googleapis.com

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by stashpro on Tuesday November 04, 2014 @05:31AM (#8555883)

nice

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by agumonkey on Tuesday November 04, 2014 @06:34AM (#8556042)

I really wonder how this 10KLines is handled by the network subsystem. Is it compiled into a big regexp ? or another form of compacted runnable logic ? or at which point does it slow local name lookup.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by jacquesm on Tuesday November 04, 2014 @08:22AM (#8556340)

At a guess, by doing a match first on a substring using a hashtable. That way if something is a 'candidate' you can hit a more expensive datastructure to figure out if you really have a hit without burning a lot of cycles.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by agumonkey on Tuesday November 04, 2014 @08:49AM (#8556441)

This kind of long blacklists seems so fitting for Tries. I should grep *nix kernels (I wish I had windows >Xp source too) I guess.

Linux (well, glibc, from http://unix.stackexchange.com/questions/81979/how-does-etc-h...):

http://repo.or.cz/w/glibc.git/blob/HEAD:/nss/nss_files/files...

Windows :

#tbd

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by mw6621 on Tuesday November 04, 2014 @06:25AM (#8556018)

Very interesting. Please do consider putting the extension up somewhere!

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by eksith on Tuesday November 04, 2014 @06:28AM (#8556024)

This list is updated more frequently and is what I use : http://winhelp2002.mvps.org/hosts.htm

There are a few things to take out to make certain services I like work (E.G. Hulu), so I usually try it in sections to see which services break and comment those.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by troels on Tuesday November 04, 2014 @05:11AM (#8555826)

Sounds interesting - care to share?

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by viraptor on Tuesday November 04, 2014 @06:04AM (#8555974)

Do you do anything about plugins sharing your fonts list? I found it to be the least known / most creepy way to identify a big number of users. Unfortunately i don't know of any extensions that would block that information.

[ Parent | on HN ]

Re:The Guardian Is Being Swamped with 'Dark Traffic' by jacquesm on Tuesday November 04, 2014 @06:23AM (#8556015)

Noscript will do that for you. To access the available fonts list you need to run a bit of js. So blocking javascript will take care of this.

[ Parent | on HN ]