News for hackers. Stuff that matters.
Re:Google Bug Bounty – The $5k Error Page by jcims on Thursday May 18, 2017 @06:00PM (#14371464)
Having worked on a large bounty program myself, and having at least one thing blow up because I dropped the ball on a response, I'll just say that the front-end aspect of it can be extremely chaotic. This guy seems like he's pretty polite and patient, which you generally try to reward with a rapid response and high touch, but sometimes you can get overwhelmed with a burst of reports, distracted by problematic reporters and bogged down by working the bug through the pipeline.

There are systems and processes to help with all of this of course, but at the end of the day it's still a pretty tricky job to get perfect all the time.

Re:Google Bug Bounty – The $5k Error Page by daddyo on Thursday May 18, 2017 @06:04PM (#14371488)
What's the general signal to noise ratio for bug reports?
Re:Google Bug Bounty – The $5k Error Page by dsacco on Thursday May 18, 2017 @06:10PM (#14371518)
About 10:1 noise:signal.

This comes from a variety of experiences: I used to manage a bug bounty for a mid-size company on Bugcrowd; in 2014 I surveyed people managing a bunch of programs across different sizes; I've participated in bug bounty programs for companies of different sizes.

The more you offer for rewards and the more recognizable your company name, the more you will be spammed by people submitting reports like (I kid you not): "You have the OPTIONS method allowed on your site this is really serious." The last time I looked at the numbers, Google had over 80,000 bug bounty reports per year, with about 10% of them being valid and maybe another order of magnitude being high severity (I'm fuzzy on the last bit). It's probably over 100,000 per year at this point. It's not uncommon for recognizable but smaller companies to receive one or more per day.

I'm aware of full-time security engineers at Facebook and Google who do almost nothing but respond to bug bounty reports. It's a lot like resumes - people who have essentially no qualifications, experience or (most importantly) a real vulnerability finding will nevertheless spam boilerplate bug reports to as many companies as they can. Take a look at the list of exclusions on a given program - you'll see that many of them explicitly call out common invalid findings that are so ridiculous it's kafkaesque.

HackerOne and Bugcrowd provide a lot of technical sophistication to prime companies for success, but there is an organizational component that is very difficult. If your program is very active, it requires dedication to tune it so you're not flushing engineer-hours away responding to nonsense. This is not to say they're bad - quite the opposite, I think they're fantastic. But I generally recommend smaller companies set up a vulnerability disclosure program through a solid third party, and do so without a monetary reward until they can commit to dealing with a reasonable deluge of reports.

Re:Google Bug Bounty – The $5k Error Page by thaumasiotes on Thursday May 18, 2017 @08:53PM (#14372274)
My favorite bug bounty report so far read, in its entirety, "try it ASAP".
Re:Google Bug Bounty – The $5k Error Page by arkadiyt on Friday May 19, 2017 @01:43AM (#14373283)
I've received reports for things like "source code disclosure" where they link to our jQuery.
Re:Google Bug Bounty – The $5k Error Page by illumin8 on Friday May 19, 2017 @12:20PM (#14377126)
LOL - I'd like to report that I was able to download the entire source code of your website by right-clicking and selecting "View Page Source..."
Re:Google Bug Bounty – The $5k Error Page by thaumasiotes on Friday May 19, 2017 @10:45PM (#14380790)
If only that were true... modern web pages frequently have basically nothing of any value in the page source; it's all dynamically loaded.
Re:Google Bug Bounty – The $5k Error Page by sundvor on Friday May 19, 2017 @01:32AM (#14373258)
~10% valid submissions still sounds like a fantastic number to me. Sure you have to sort out the bad ones, but it's still a solid stream of valid reports.
Re:Google Bug Bounty – The $5k Error Page by jcims on Friday May 19, 2017 @08:48PM (#14380390)
It's a pain when you're in the thick of it, but it really is a great way to round out your security program. There's an astonishing number of incredibly skilled and motivated folks out there, and a well-run bounty program can create a nice symbiotic relationship that benefits both.

One other thing that never really gets any press is the fact that a good chunk of the folks sending in reports are young people in impoverished nations. Some of them can be pretty tricky to deal with, but if you hold a hard line on professional expectations you can see them flourish in pretty short order to be some of the best reporters out there.

I only spent a short amount of time on the program I was with, but it was very rewarding. A+++, highly recommended.

Re:Google Bug Bounty – The $5k Error Page by sundvor on Tuesday May 23, 2017 @03:35AM (#14399529)
That's great about the young people! Thanks for relating this.
Re:Google Bug Bounty – The $5k Error Page by jcims on Thursday May 18, 2017 @08:13PM (#14372101)
Whoever runs this definitely works in a bug bounty program:

https://twitter.com/cluelesssec

Re:Google Bug Bounty – The $5k Error Page by martenmickos on Friday May 19, 2017 @02:24AM (#14373395)
Generally in the world of bug bounty programs, the signal-to-noise ratio (SNR) is around 10-20%.

Even at this low rate, it is not too bad. Let's say you receive 10 reports. You can relatively quickly identify the 8-9 noisy reports to find the 1-2 valid ones. Of course, a higher SNR is always better. It saves you time and effort.

On HackerOne, the average SNR across all programs is over 30%. The platform can automatically filter out certain reports that are duplicates or out of scope.

The platform maintains an average signal rating for each hacker (aka security researcher). Companies can limit access to their programs to hackers with a certain signal or higher. This will significantly increase SNR for the program.

Companies can also opt for a HackerOne program with triage included, in which case the SNR rises close to 100%.

Re:Google Bug Bounty – The $5k Error Page by txutxu on Friday May 19, 2017 @03:26AM (#14373595)
> The platform maintains an average signal rating for each hacker (aka security researcher). Companies can limit access to their programs to hackers with a certain signal or higher. This will significantly increase SNR for the program.

So if a new user of the platform, finds a valid or high impact bug, will be unable to report... less noise but a high value bug unreported in that case...

Re:Google Bug Bounty – The $5k Error Page by robbiemitchell on Thursday May 18, 2017 @09:56PM (#14372505)
Is there a ticketing system of some kind in play there? I imagine there would be steps like "Respond to user" before resolving/closing.
Re:Google Bug Bounty – The $5k Error Page by dingaling on Friday May 19, 2017 @02:24AM (#14373396)
I'll don my corporate hat and say: those are completely unacceptable excuses for poor client communication.

I'd perhaps accept 'I was so deep in the code!' once from a very junior developer, as a learning experience.

To the person that reported a bug, that one report makes or breaks their entire opinion of your organisation. We lost customers because of poor communication, and on the other hand made some very happy repeat-customers even when we had to say 'we can't fix that yet' - but they were in the loop for the whole process and understood why.