News for hackers. Stuff that matters.

A comment on Company withdrawing from Facebook as analytics show 80% of ad clicks from bots. Parent comment

Re:Company withdrawing from Facebook as analytics show 80% o... by rscale on Monday July 30, 2012 @04:24PM (#4314492)
One as-yet-unmentioned technique is to adjust a bot score by taking an OS fingerprint, and comparing that to the listed user agent.

It's not perfect for a variety of reasons, but I found it to be a useful input for a similar bot detection problem. That said, this was a long time ago so I'd need to re-run some experiments to see if the hypothesis remained valid.

For those not familiar with OS fingerprinting, it's a method of looking at the details of a connection and determining which operating system is most likely to have created packets with those options and flags. In BSD, it's built into pf.

[ Parent | on HN ]
Re:Company withdrawing from Facebook as analytics show 80% o... by tripzilch on Monday July 30, 2012 @07:01PM (#4315147)
On the packet level ... hm that's pretty clever, definitely not trivial to fake, either.

Just that if your bot runs from, say, Windows 7 and it spoofs an IE8 user-agent header, or even runs by directly automating IE itself, how do you detect it then? Both of those scenarios are not unlikely at all.

[ Parent | on HN ]
Re:Company withdrawing from Facebook as analytics show 80% o... by rscale on Monday July 30, 2012 @08:34PM (#4315451)
If the bot spoofs a user-agent that is viable for the OS, then you need to rely on other signals. It only works as a component of a defense in depth strategy.
[ Parent | on HN ]
Re:Company withdrawing from Facebook as analytics show 80% o... by dredmorbius on Monday July 30, 2012 @05:42PM (#4314838)
Interesting. Any Linux equivalents or Java classes which offer similar capabilities (we're using a Java-based application server / webserver).
[ Parent | on HN ]
Re:Company withdrawing from Facebook as analytics show 80% o... by biot on Monday July 30, 2012 @07:01PM (#4315149)
It works off of checking the SYN packet [0], which happens at the transport layer [1]. Once it gets to the application layer the information is not available unless the transport layer stores the information and provides a method to query a given connection.

[0] http://www.openbsd.org/faq/pf/filter.html#osfp

[1] http://en.wikipedia.org/wiki/OSI_model

[ Parent | on HN ]
Re:Company withdrawing from Facebook as analytics show 80% o... by dredmorbius on Monday July 30, 2012 @07:48PM (#4315295)
I suspected something like this, but was hoping enough of the signature might survive for a daemon (or Java class acting as one) to take a peek.
[ Parent | on HN ]
Re:Company withdrawing from Facebook as analytics show 80% o... by rscale on Monday July 30, 2012 @06:54PM (#4315124)
The linux equivalent is p0f (http://freecode.com/projects/p0f)

I can't speak to Java classes, but they'd need to be dealing with raw packets which isn't an option in a typical stack.

[ Parent | on HN ]