I'm a burnt out security focussed software engineer of 9 years. I worked at 2 well-known Bay Area tech companies. The first 7 years were incredibly rewarding, but now I'm taking a year off the job market. At the end of my tenure, I felt like I was a few days away from being committed to an insane asylum.

Doing anything security related is amazing if you work with the right people, but more often than not the amount of snake oil and petty politics is intolerable, more so than in any normal software engineering gig.

I know a few incredibly competent ex-Uber security software engineers at Uber (disclaimer: I never worked there), who reported to Joe Sullivan, a former federal prosecutor with no technical knowhow whatsover, now facing criminal charges [1]. They're very traumatized from the incident. Trust me, this sort of sleaze is on par for the course at security orgs in major Bay Area tech companies.

[1] https://www.nytimes.com/2020/08/20/technology/joe-sullivan-u...

Wow, I didn't know about the Uber cover up. Thanks for sharing.

Thanks for sharing that. Aging vulnerability researcher and reverse engineer here and it is both amazing and disheartening to see what most of security work has turned into.

Layers and layers of policies, procedures, and abstractions that mean no one is ever accountable for anything and anyone who knows the details is marginalized as being too much “in the weeds” so they’re left out of the conversation.

Are all those mumbo-jumbo-sounding security certification acronyms part of those layers? :) ICSSP? and such.

No, nobody tend to care about certifications in the field.

could you list some example technologies that exhibit these issues?

CIO in financial services here. Most security activity is/will be driven be external forces. Auditors, regulators, etc. This tends to drive organizations towards “checklist” security. Real security professionals know that it’s not really about “can we check the MFA box” it’s about HOW we implement MFA. Unfortunately, that discussion gets squeezed in favor of “we have to close this finding by the next risk committee meeting...”.

> This tends to drive organizations towards “checklist” security.

I have seen the pharmaceutical manufacturing equivalent of this. No one actually cares about microbiological surveillance of drugs for animals, they are just forced to pretend to because otherwise they get in a lot of trouble.

Exactly this. Software development and security are basically the same thing. If you’re not writing secure software, and you don’t understand enough about network layers to secure them, then you have a lot to learn just to get your knowledge up to par with a lot of devs/arch’s/“security people”

>Software development and security are basically the same thing.

There is an entire universe worth of knowledge in the security space that has nothing to do with software development. I'd agree that 98% of the security world is "bullshit", but one of the reasons every company has such awful security is specifically because too many people think "software development" == "security", and this narrow mindedness means they end up failing to accomplish any of the security pillars that have nothing to do with software development.

There is a lot of overlap specifically with secure development and software development, but saying "software development and security are basically the same thing" is naive. I like to compare it to a company's legal team: if you're going to court over a technical topic, you definitely want some people on your legal team that have software experience and you definitely want your general counsel to be technically educated, but that doesn't mean you're just going to take a software developer and make them your general counsel. You still need an actual lawyer, because there is an entire universe of law-related knowledge that you need that a software developer doesn't know.

Exactly. Small stuff like having separate cloud accounts for different environments like prod and dev, enforcing MFA and password rotation, having separate ACL roles for different services with minimal permissions, basic antiphishing training, and so on are all low hanging fruit that few software engineers have the knowledge of and even fewer have the political capital to implement unless its handed down from the ops team.

Instead defense in depth works best when the software engineers are treated as benign manifestations of hanlon's razor.

I thought password rotation is considered outdated.

It is. The watershed publicity moment was in 2010, when the research paper came out.[0] It took another 6-7 years before the news percolated high enough in the political chain, and now both NIST and NCSC recommend against the outdated practices. And the infosec community had been arguing against the stupidity for what, 20+ years?

Now, there is a place for rotation - when the so called password is in reality a shared secret. (Eg. the secrets in payment gateways.) Such things need to be rotated, because the basic assumption is that they will be compromised. No matter what you do, someone will copy-paste a long-lived secret to the wrong place at some point.

0: https://www.cl.cam.ac.uk/~rja14/shb10/angela2.pdf

Thanks. What I find the worst about 'must not be similar to past pw' is that you have to store the pw in plain text somewhere (or at least retrievable).

An alternative to storing the pw in plain text is to ask the user to provide their current password at the same time as the new password. The password change routine can then check the current password is correct (which protects against the threat of an attacker coming across an unlocked terminal with a logged-in session and changing the password) and provides the current password against which the new password can be compared.

In my experience, security roles have more to do with the practices of risk management than anything else. With security policy and compliance roles also requiring an element of jurisprudence.

Computer Science is Security... Bruce Schneier agrees with your view: https://www.schneier.com/blog/archives/2016/06/computer_scie...

My perspective as a software engineer is that security is a much more technically challenging field that SWE. Is this true? Is security just smoke and mirrors? And why is SWE so much better remunerated?

Because so much of the function of the department is social. They create rules and communicate them. Their approval is often required on projects but they add little value otherwise. The position exists to provide cover.

Often we think security as a team in a constant battle with hackers and rival corporation forces but in reality you will mostly be spying on employees and setting policy.

It's better remunerated (to the extent that it is, obviously some people make fortunes in security) because products make money and security costs money.

As a security engineer, i dont think so. Its definitely different, and because there are less of us it makes it seem we are more specislized. However i've met tons of SWE working on hard problems that don't make sense to me, and can code circles around me. The level of technical challenge in both fields depends what you are working on. Plenty in security is also trivial.

Same for security and audit processes. Most of these are a largely pointless waste of money and time. Some are well-intentioned, but utterly impractical to the point of questioning the well-intentioned part (e.g. "all releases of all software needs to be audited").

Not to mention many auditors have minimal understanding of the stuff they check on. On a number of occasions I had to explain how the system worked to the auditor, and what's worse - they seemed to be OK with me just explaining that, without any double-checking.

That's where they need better-educated, or more capable auditors.

I used to do PCI auditing for organizations that handled credit card data. The course was a two-day joke, and they relied at the time on some stated level of prior experience.

Clearly, the class was intended to give you knowledge of what's required, and how some processes worked, but you were expected to know the security stuff first.

While I was not a huge fan of the organization, I was somewhat more satisfied when I could help the clients with reasonable analysis of their controls, and where they'd fall down (regardless of tick-box compliance.)

> 98% of the security world is complete and utter bullshit, and no one is interested in actually securing anything

As someone who did it for six years before leaving for better things, I couldn't agree more. It really is a circus that sells feelings at best.

It’s a joke, but it can pay really well.

Better also enjoy reams of eye glazing paperwork

The interview process is also much easier than a SWE

From an executive management point of view, security incident insurance is often seen as cheaper and safer than investing in actual security. That often leaves compliance as the only real security emphasis.

Yep, and then when the insurer declines to cover an incident because the company neglected proper security, you do NOT want to be the person with "security" in their title.

I was in the same position as OP...subscribed to 2600 as a high schooler (maybe I'm on a list)...then after college became a software engineer w/ interest in security. Even got a Security+ certification 10 years ago. Afterwards, I learned most of the security work is policy...at least, that's what people that I knew in the field told me.

Now I'm still a software engineer and get the occasional rush when I need to fix something that our Fortify scanner picks up (rarer now, since we rely so much in frameworks nowadays)

> EVERY bank has terrible 2FA practices? It's because they don't care.

Good point, but its more they tradeoff losses vs convenience. HSBC pushed a hardware security fob for every time you wanted to check your balance. It was a PITA until they relented. If they kept requiring the device for every log in I'd have closed my account.

Relevant blog post: https://www.go350.com/posts/they-want-us-to-be-compliant-not...

Good advice.

Security is a vertical that tends to attract really really smart people and people looking for a landing zone, functioning as unaccredited auditors or folks performing monk-like transcription of NIST or compliance guide. There’s no middle.

The only really technical person I know who is really happy in the space transitioned into a security-focused solution architect type role with a VAR. But he is one of those rare people who is very deep in a few tech disciplines AND loves engaging with people.

I've worked with a number of people who land in "the middle" IMO. These are application security engineers hired to staff the security teams at companies building normal b2b or b2c products. Their job is a mix of hands-on fixes in application codebases, triage of reports from external researchers, and scripting/building their own systems. They're coders who remain practical, not world-class cryptographical researchers nor spreadsheet jockeys, and they're darned useful to have in place at companies where you can't expect every developer to maintain up-to-date deep domain knowledge of the security landscape.

I've had that role twice! For me, perhaps I just haven't found the right place, it starts out as the best job I've ever had and then ends up being a terrible slog. Developers stop caring, management does the math and finds it easier to have you on staff but doing nothing really important because you keep making the dev team fix their SQL injections or XSS and they miss features and blame the fixes that you pushed for, and then you spend 90% of your time chasing dev's to update their libraries and you want to harm everyone. It's just a tough road to work if you are at all a creative developer that wants to see something you have built. I REALLY wanted to love that job but even for good companies it's very hard to find a good situation in my experience.

Same here. You're just outnumbered, and because of that you're often seen as having unreasonable requests or point of view. I don't think anybody who works in a security team is happy, it's a shit job honestly. The less you do, the happier your devs are. And at the end of the day, you look back at the years of work you've done, and there's nothing to show. No building, it was just about pushing through friction.

That’s fabulous! Seems to be a hard role to find in many companies.

> Source: I did penetration testing for four years, also served in a cyber position in the military. What a giant waste of my time that whole effort was.

But the money must've been really, really good.

Security jobs usually pay worse (if not much worse) than SWE of same or even lower level.

Why functional languages? I have some experience with Haskell via a class I took in uni, although we didn't go into detail re: security.

Not in infosec but probably b/c you‘d have to jump some hoops to get your code to have "side effects" with functional.

So from a security perspective it's way easier to review/audit functional code.

Yes in general a pure function that takes in data and returns a result without reaching out of the function itself (in general that's a side effect) is usually more secure. There are certainly exceptions to that rule, like parsing data formats however. Also most pure functional languages make you isolate side effects which makes security auditing easier. For example I was an application security engineer for both a ruby company and an elixir company, both of which I'm sure you have heard of and both had excellent developers. The ruby shop (which in general I'd say had a higher talent level) had far more security issues in the code than the elixir team. Both had vaguely similar use cases and threat modeling also. I do believe that functional programming tends to lead to more secure code.

Why was it a waste of time? If you found just 1 security flaw that could’ve resulted in a ransomware attack, wouldnt it have been worth it?

Here is an example from my career: I once found a SQL injection at a large internet company (I won't say which one) that dumped user accounts including hashed passwords, email addresses, comment history, and a ton of even more sensitive data. The management of this company did the math and said the risk of someone maybe finding it vs the cost of the dev team missing a feature deadline made the vulnerability less important to fix. The sad part is that they were proved correct. The dev team finished the feature and then merged in my fix for the injection (it was a relatively large fix involving a large constructed SQL statement and I wrote it in my off hours because I couldn't just leave it). No one found the issue and the feature landed them new business. In the end they got what they wanted, the fix and the new business, because I gave up extra hours. That's when I decided that app development better suited me.

Depends.

Security flaws are not created the same. Mostly they are hypothetical. A buffer can over flow and smash the stack, but what then?

A security flaw that costs a million dollars to find and has very little chance of being actually used is not worth finding. But when you start looking, you do ot what you will find.

Like most things, ity is a trade off

Thanks for sharing. Do u think reverse engineering in a certain field (e.g. Hardware or Windows applications) could be a better path?

Also, RE skills themselves are pretty useful for ordinary SWEs, it's worth pursuing even if you are not going to work full-time in it.

yes, as you'll be working on a consultancy most probably

> for firms that write in functional languages

What does the language being a functional language have to do with it?

If I had to guess, they're likely referring to the lack of side-effects in some functional languages. This makes it a bit easier to formally verify the correctness.

What's funny is that some of these languages lack types, which makes it a whole lot harder to secure.

So it's not just about being functional.

got any fun Red Flag stories?