That's actually an interesting proposal-- 100% of the time when I click on an ad, I didn't mean to and I try to close it before it has a chance to annoy me.

Probably doesn't make sense on a large scale, but it's something to consider.

Someone's grandmother miss-clicking and then mashing back is about as useful to the advertiser as a bot. conversions are where it's at. Poor quality traffic / bots / it's all the same. the advertiser quits paying

They reinvented the access log? By hand? Are people really that detached from how servers work?

Anyway, it's pretty irrelevant whether they added logging in their app (one line of code?) or enabled their webserver's built in logging.

Since the user is on facebook they'd have facebook unblocked, but noscript still blocking everything else.

How would you discern a noscript user from bot?

"...in all of our years of experience, only about 1-2% of people coming to us have JavaScript disabled, not 80% like these clicks coming from Facebook."

For example, target something for the Reddit.com users and you'll see 90%+ block ads.

With facebook granular targetting you could very specifically end up with a target segment with a large proportion of noscript type plugin users. Without more details on their campaign I would not be so quick to place blame purely on bots.

And even among the most technically literate groups, I'd be amazed if you saw 80% use of noscript. So many sites require javascript that it's easier to browse with it enabled.

There are a few tricks to do that, described on the old ha.ckers.org weblog, or perhaps on the related sla.ckers.org forum. No idea if they still work, though ..

You could always see if you could trigger NoScript's built-in XSS or frame hijacking protection :) Bots will not notice, and users get a scary warning message and believe your ad is trying to hack them. Not good for your conversion rates, but a pretty sure way to tell them apart.

Or they might have indeed reinvented it. Kids these days and their Heroku! Back in my day we'd process each HTTP request by hand. And we'd do it uphill both ways in the snow.

Most of the parsing required for the logfile analysis I describe is written in awk, and goes through roughly 1 million lines of logfile in about 75s on a commodity 2Ghz Intel Xeon.

Pre-caching of host lookups is done via xargs and a suitably high '-j' value. Running on a pre-sorted and de-duped list of IPs, this takes another few minutes. Lookups are read into hash tables for faster processing in the main awk parser (avoids system calls, DNS latency, and especially negative result caching failures).

It's fast and simple. Not a 100% solution, but quick to add bits to over time as new needs arise.

It was the proper add-on to the previous comment kids today don't learn awk. Becuase they have Perl, Ruby, and Python which are supersets of the unix command line tool set. Awk was the king in the years before Perl.

I never really learned to use cut, so I sitll use awk for one-liners the deal with columns in delimited-data text-files.

Depends how far they went profiling their visitors, it clearly says in the article they were tracking if js was enabled, so yes, you'd need to roll something or another by hand. GA relies solely on js for example.

And then compare in the log.

I am not familiar with how things work on FaceBook, but I would think that FaceBook users do not have access to the logs that FaceBook's servers write.

And so it seems that Facebook could be proven to be the new groupon.

I would absolutely not be suprised if the hits were FB bots - however I'd expect they wouldn't be that directly stupid so as to have an Ad-bot net in house. Surely this is some service they outsource for plausible deniability.

Something they would have learned from their relationship handing all user data over to the NSA/CIA.

EDIT:

I'd like to make a prediction on how this could potentially play out:

1. FB ignores them. They have nothing to do with this issue and they know the bots are tainting traffic - but they can't/wont do anything about it because it boosts their revenues.

2. FB may or may not be involved in the ad-bots, but, they will drop their $ contingency on ad-rev for the name and payout these guys to shut them up. Their stock was improperly priced and taken a beating. They don't want this coming to light in a large way as it will have a negative affect on the perception of their business model. They hav to be kind of careful of the Barbara Streisand effect here too...

3. Many more people will reveal the same results with empirical testing and it will be revealed that FB earnings are 80% over inflated. Their stock will drop to the predicted real value of <$10... maybe hit that actual projected number of $7.00 - Zynga will BEG FB to purchase them as they are now a penny stock and they can't sustain.

Also given FB users' aversion to ads this whole issue could be explained by users just clicking the back button. Ad supported sites have trained users to do three things very well: install adblock, habitually ignore ads, quickly click away from any interstitial or unexpected ads like on Hulu.

Another note on the "back button": I recently discovered the awesome power of "three-finger swipe back button" on Macbook trackpad. So it's possible users can "click" back even faster than before. I can and before the swipe I was well-trained in the art of "CMD back arrow" :-)

Also, FB may not have much of a choice to comply with those agencies, or find themselves pissing off a lot of high level people by refusing to comply with government agencies. But outsourcing / tolerating a network of bots to vastly boost revenue is probably considered fraud, no?

It's not what people brag about, it's the social graph itself that's useful. All the other things like photo-tagging are extra candy.

More info at http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/al... and http://nplusonemag.com/leave-your-cellphone-at-home . Sorry it's late out here, or I'd copy/paste the relevant quotes for you--look for the bits about data mining and machine learning, especially in the 2nd article, everything is a signal and they do have the computing power to process it all, apply Bayesian belief nets, algorithms can pull intelligence information about you from the local shape of your social graph that you don't even realize can be inferred from it (humans are not very good at reading complex info from big graphs much further than simple "friend-of-a-friend" relationships).

And that they are doing this using experience from working with the NSA/CIA in order to cover up the fact that $800 million dollars of Facebook's yearly profits will disappear overnight once people realise the truth. And as such they are willing to pay out the ad-bot companies in order to hide this conspiracy.

Right.

I do not know is it applies to FB, but those who have like 100k bots have some power too.

[0] http://imgur.com/Vvkbx screenshot in russian

Edit: spelling Edit2: proper link

What about the farking article? Where they proved that 80% of their clicks were from bots. Then several others confirm they ahve the same experience?

What about those facts?

I also speculated on what FB is doing. Them giving user data to the NSA/CIA - that is a supposed fact, sure, but I am not the only person who believes this.

Am I extremely distrustful of Facebook? Of course! Maybe you have not followed their track record or the character of their founder.

> I also speculated on what FB is doing. Them giving user data to the NSA/CIA - that is a supposed fact, sure, but I am not the only person who believes this.

Great argument! A lot of people believe it, so it must be true! Are you for real?!

And the backdoors in Cisco's equipment that are a requirement by the federal government?

Maybe you thin this shit is "conspiracy theory" and if you do, you're simply a naive fool.

Look around you at what governments are doing. The NSA has records on everything you do online. They had a system in 2005 which could trace communications between users to 6 degrees, automatically.

I know a couple of guys like you in my local activist community, who take a very hostile "I know the truth and you're all fools!" attitude, despite the fact that their audience is mostly very sympathetic to most of their assertions. We know spying on the Internet takes place; HN is full of cantankerous old Internet geeks who've seen, first-hand, plenty of examples of the state (whatever state you may choose, as it happens all over the world) behaving unethically on the Internet, spying on people and punishing people for things that shouldn't be crimes.

But, your paranoid approach is counter-productive. You might as well be working for the people you claim to be afraid of, for all the good you do (negative good; you're convincing people that the folks who believe the government is spying are all paranoid nutjobs who scream at anyone who has the gall to mention other possible explanations).

So, let's review:

1. Just because spying has taken place, and is currently taking place, and may even have the complicity of facebook in that spying, it does not mean that facebook is running a botnet to steal advertiser dollars. The simplest explanation is that facebook looks the other way while others run the botnets. facebook wins (a lot, as long as most advertisers don't know it's happening), botnet owner wins (a little), and the advertiser loses. But, there are other plausible explanations, including incompetence.

2. When you paint things in a "Either you accept my theory in its entirety, or you're all idiots", you force people to choose a side. Nobody wants to be on the same side as an asshole, so you force them to choose the other side. You make people who may even agree with you (to a greater or lesser degree) to begin to formulate plausible reasons for why you're wrong about the crazier stuff you're spouting...further convincing themselves that you're entirely wrong. The best you can hope for is people ignore you and don't have the chance to be inoculated against your ideas; having you as their first exposure to these concepts guarantees they will be less likely to believe them in the future, even if they come from a more credible source. Humans are funny creatures.

Thus, I would point out that there are some really naive and stupid people on HN that have zero grasp of effective argument, persuasion, and even basic logic.

You might be well-served by reading about non-violent communication: http://en.wikipedia.org/wiki/Nonviolent_Communication

Edit: Removed the word "schizophrenic" as it was an insensitive use of the term, and was counter-productive to making my point.

In my defense, schizophrenia runs in my family, and I'm very familiar with it...I don't think of it as an insult. But that's a local custom in my family that I shouldn't think follows in the rest of the world.

Really, now I am a paranoid schizophrenic?

Just because I make claims which are readily confirmed and were completely available in the media - even the EFF filed suit on the AT&T events...

Yet, for some reason, it is my responsibility to educate everyone every single time someone new comes along who hasn't been following these things closely.

Now I am a paranoid schizophrenic?

Also, its a strawman to focus on my tone, rather than content. You're trying really hard to be overly pedantic and, frankly, an asshole.

Thanks for the link on communication.

You obviously have a vested interest in seeing your "message" be received, so stop ignoring what people are telling you about your tone and change it. Figure out what it is that people do listen to, or else you have no one but yourself to blame when they take issue with your tone.

To wit: Your tone is defensive. Pejoratives and cursing undermine your expression. What you are effectively telling people is that your thoughts are not important enough to merit self-restraint. Detailed explanations following assertions are also helpful.

FWIW, I work in the performance marketing industry and, IMO, the most complicity facebook could be said to have in this issue is in not placing a sufficiently high priority on preventing bots from clicking on their ads. Even if it is a sizable project there, they represent such a large target that the difficulty of the job becomes much greater. I see no salacious story here, other than a company found itself unable to optimize a campaign into profitability, which I think says more about them than facebook. Ho hum, find a different traffic source and move on.

However, I will point out that with respect to your comment on FB's complicit actions due to the daunting nature of the problem, this does not take into account their other actions of a 24K ransom on the domain.

Everyone can argue in any direction they want - but neither me nor anyone else is really going to know until we get further down this path...

When I was talking about China hacking lockheed as far back as 2005, everyone said I was nuts! (I freaking worked at lockheed!)

There needs to be a better way to log and track this stuff so that we can point people to some sort of Tyranny Wiki.

I use noscript. When I enable it for visits it's usually by enabling the domain itself and leaving third-party scripts disabled. Depending on your area this could be significant.

Also the possibility of blocking of the scripts by other elements not loading - for example if the browser can't parallelise the requests for some reason and a preceding request can't be handled then the page might be displaying well before the script is loaded, especially if it's loaded at the bottom of the markup.

http://pastebin.com/raw.php?i=HHeY5nP1

$ curl -s http://pastebin.com/raw.php?i=HHeY5nP1 | md5sum 7b69269bd0e25daa8e8b171e5d513011