As much as I admire people who show the courtesy to the trash bin that will eat their report, before any human reading it, of not feeding it "bad" reports: The OPs point of "sense of false security" unfortunately already sets in once someone is hired to "take care of security". This anecdote from about 2000 illustrates that:
A friend was hired to do black box penetration testing on the DMZ and internal network of one of the largest travel agencies in Europe. He, of course, found a lot of problems. He wrote a nice report. Like really nice. Point for point "this is your problem, this is what you have to do right now and this is a user-friendly policy you could implement to prevent such issues in the future." It was very pleasant to read, he knows how to handle language. And somewhere on page hundred-something "the first one to claim it, will get a bottle of champagne!".
The bottle was claimed a few months later by someone 3 levels above his position. By some high-level manager who did not know anything about IT. But he was the only one to read it (and he only claimed the bottle to figure out whether anyone below him had actually read the report). Two years later company politics allowed those 2 illiterates between my friend and the one manager who could read to be removed from the company. My friend was hired as their chief of security. His first task: work through his own report to fix the 90% that had not been fixed since he wrote it 2.5 years before (the report listed passwords to core routers in plain text as examples for "very stupid passwords" - they all still worked).
[1] It didn't help my perception of the report when it screamed that "ICMP echo was enabled and nefarious scalawags might be able to do unspeakable acts against our computers, best cut the network cable" type of advice [2] (yes, I know port 443 is open! WE'RE A WEB HOSTING COMPANY SERVING UP COMMERCIAL WEBSITES! ARE YOU PCI AUDITORS STUPID?)
[2] Okay, the ICMP echo thing was reported, but did not need to be disabled to pass the audit. If so, why even bring it up?
I personally organized my report into three sections, which seemed to work well. Clients seemed to enjoy the formatting:
1. Executive - Summarize everything in one page at a high level. You could skim it fast if you chose to. Highlight potential negative business impact of each finding.
2. Management - A little more detailed. 2-3 pages max. Most severe findings at the top and recommended action for remediation.
3. Narrative - This is the bulk 80-90% of the report detailing your step by step process including screenshots so that if someone wanted to duplicate your findings they could.
This was for using citrix boxes to provide saas to a client
Break it down to a 4th grade level, if you can't, you likely don't understand it yourself.
I would agree that in general certs have too much weight, but the reality of it is, as it stands, they're the closest bridge / standardization that the market has built for non-Security ilk.
Also, Security is not IT. That's another thing that needs to change.
As for OSCP, I think it's great and out of all the certs i've taken for various reasons, it's the only one I felt challenged with, in a good way.
Do you mean that
Security is not currently IT and it should be
or do you mean Security is considered to be IT and it shouldn't be
? The amount of stress that you mean to put on the word "not" doesn't come through very well in this medium.Security should have an advisory role to the corporate governance folks who maintain policy... usually the lawyers.
Equally importantly: memory corruption exploit development and SQL injection are different skills, and most people who do SQL injection don't need proficiency in "buffer overflows". Why is superficial coverage of "buffer overflows" part of the rubric for that certificate? I don't know, and I don't know that anyone else does either.
Is there a single coherent security certificate anywhere in the industry? I'm interested in examples.
I don't think Offensive Security is trying to pump out exploitation experts from their entry level cert program. Maybe the higher levels OSCE and OSEE. The intro cert emphasizes breadth over depth. It felt a lot like a cert built around the Exploitation Hackers Handbook.
I think you're thinking of the certs in the wrong light. They are meant to validate baseline knowledge and proficiency, not mastery. If you want to validate mastery you need to look at the persons personal record and work product.
Total tangent but, I am absolutely grossed out by "cyber" winning out in the name game. Who let the DoD drive that? Damnit!
It's the only one I ever thought might be valuable since it combines specific knowledge with some pentest against real systems. Curious what you think as you're more qualified to assess such a thing given all your security evaluations and hiring those people.
The leet kids, a minority few and the rest naïve, I would bribe while they whittled away at online CTFs and MicroCorruption and irritate them with mediocre questions until they tuned me out. I did not care for tools; approach and mindset are order of magnitudes harder to explain.
I thought you could be a wakeup call had you told them all the certs pro se is a waste in a program that pushed that nonsense. I talk trash of my certs and skills the whole time and they did not get why.
I know you're busy, but I've read your blog and you're preaching to the choir. Starfighter folded, but I would pay for you to test me as a customer and find a mentor to answer my stupid questions that I would pay handsomely for the privilege. I feel I'm not the only one, if you get tired of NCC, that would be amazing!
A well written report that speaks at the right level to its audiences will generate more proactive security activity than a terse, passive voice bomb of dense technical information.
Edit: And, in contrast, I've met people who I had a hard time communicating with in spoken English but, had no problems with once we did our communications in writing or with a written point of reference.
I vehemently disagreed, pointing out that those of us with meaningful degrees from well regarded programs shouldn't have to get one, but in the end I complied to keep the peace.
Have to say I enjoyed it at the time and even learned a thing or two. I wasn't exactly new to offensive work, but was nice to get things sharpened up and familiar with a few different approaches. I remember being a bit worked up about the final challenge at the end of it that I took the day off work, but iirc I was able to finish in just a couple hours. It wasn't the most cutting edge content at the time, but it was well organized and ensured that the person being certified could demonstrate some level of practical application and that is far better than most certifications of any sort.
There are a couple of exceptions, of course. OSCP is a good certificate to have. To pass the exam, you are required to not only demonstrate proficiency in several areas (i.e SQL injection, buffer overflows), but you must also write and submit a technical report to a review team. The technical report must address vulnerability overview, impact, risk rating, reproduction steps, and more. Of course the exam isn't perfect, but it's probably the biggest test of real technical understanding and ability I've ever seen.