AVG disguises fake traffic as IE6

Re:AVG disguises fake traffic as IE6 by gm on Friday June 27, 2008 @11:22AM (#229816)

Any opinions on the best way to deal with this?

[ Parent | on HN ]

Re:AVG disguises fake traffic as IE6 by avalanche on Friday June 27, 2008 @08:48PM (#230385)

I think the solution should be fairly simple. Since AVG is not a real browser, despite telling a web server that it is, it should be easy to craft a piece of Javascript and cookie on a webpage that would return a proper result in a real browser like IE, Firefox, Safari, or Opera. If a proper result is not returned, then the web server can just do a 302 meta redirect to AVG's website as mentioned earlier. If implemented widely on web servers (say as an apache module or PHP patch) this, in effect, would cause AVG to become it's very own DDoS attacker. I don't think they would want this, but it'll probably happen if they don't re-think the way they do things.

[ Parent | on HN ]

Re:AVG disguises fake traffic as IE6 by mleonhard on Saturday June 28, 2008 @08:19AM (#230574)

How would that work? The AVG robot probably only fetches the page's URL. By the time you serve up a JavaScript, you've already lost the ability to redirect the robot.

Also, once enough people start redirecting the AVG robot to the AVG website, they'll just update the robot to ignore those redirects. It's antivirus software so it gets updated very often.

[ Parent | on HN ]

Re:AVG disguises fake traffic as IE6 by avalanche on Saturday June 28, 2008 @10:51AM (#230603)

I guess it could be done using some combination of first placing into every HTTP header: "Refresh: 2;URL=htpp://www.grisoft.com/", which would then allow a piece of Javascript on the web page up to 2 seconds to figure out if the visitor is a real browser (maybe even including some DOM checks too), and then acting accordingly (ie: sending the user to a page without the redirect header). If AVG's http fetcher doesn't execute the Javascript or fails to pass the browser test, then it gets redirected back to AVG's site because of the HTTP header.

Convoluted? Yes. Will it work? It should, unless AVG decides to implement a full Javascript engine and DOM stack in their http fetcher. Also, the initial re-direct doesn't have to go to Grisoft, it could go each time to a randomly selected site from a list of their competitors <evil grin>.

I think AVG is really playing with fire, it's really only a matter of time before things like this start popping up as defense mechanisms.

[ Parent | on HN ]

Re:AVG disguises fake traffic as IE6 by pmjordan on Saturday June 28, 2008 @03:39PM (#230723)

Presumably, AVG loads extra resources like images, CSS, iframes, <object>s etc. from your page. (if it didn't, it wouldn't be very good at finding malware, and be really easy to detect)

[ Parent | on HN ]

Re:AVG disguises fake traffic as IE6 by whatusername on Monday June 30, 2008 @03:20AM (#231781)

actually - from skimming the article it appears that it doesn't.

[ Parent | on HN ]

Re:AVG disguises fake traffic as IE6 by ajross on Friday June 27, 2008 @11:43AM (#229835)

There aren't any. The User-Agent header has always lied, at least a little bit. What AVG wants from your site is the same thing that the users do: the content (in this case, so that they can scan it). If you insist on giving them a different page (like an error page) than a user would see, they're going to take steps to prevent you from doing that.

Remember that the desired outcome here is to see fewer AVG hits, not to "screen them out" just because they're not a human being (although they are a necessary condition for a human hit coming later). Try reducing the unique URL count on your site (basically: make it more RESTful and eliminate all the query strings on broadly identical content) so they have fewer URLs to crawl. Set your caching headers properly so that they don't have an excuse to recrawl you too often. That won't force them to hit you less, but it will "help them to help you" by reducing the total bandwidth. Remember they don't want to pull any more than they have to either.

[ Parent | on HN ]

Re:AVG disguises fake traffic as IE6 by BrandonM on Friday June 27, 2008 @01:40PM (#229967)

What AVG wants from your site is the same thing that the users do

Then shouldn't AVG report the same User-Agent that the user's browser is reporting?

[ Parent | on HN ]

Re:AVG disguises fake traffic as IE6 by aristus on Friday June 27, 2008 @04:04PM (#230122)

If AVG didn't have their head up their asses, yes. Though that would make the problem for webmasters even harder. What really chews my liver is that they didn't even do it right.

[ Parent | on HN ]