From my experience; there is a huge NEED for app sec engineers, but the DEMAND isn't really there. Even if you find an organization that DEMANDs a security engineer (rare); that professional has an uphill battle to be granted the resources, time, and power to make meaningful change.

I'll second this.

So few organisations have dedicated security people at the time they need it most! Since we're on HN, it is maybe worth looking at startups, and how few early stage ones build with security in mind at the time. In regulated sectors or ones where reputation is key, it seems a no-brainer to have a security minded joint CTO/CSO to ensure the house is in order from the start. I understand the pressure and focus on an MVP, but equally I've seen first hand the cost of undoing (avoidably) bad engineering and security ignorance that had real financial cost. After MVP is validated and there are paying customers, it's really time to get security right, in my view.

I also see little demand for "real" security engineers. Most security roles I see are pretty non-technical, often doing the whole "let's turn security into a generic risk so we can hand it off to a non-expert". I don't really see how you can lead or drive security when you can't pop your way into a typical business webapp yourself, yet most of the security people I know what to learn these skills (but they're too busy firefighting for me to get time to teach them some of the tricks).

I've seen little demand or appetite for bringing together what seems to me to be a real security engineer - experienced technical developer lead, deep and broad security knowledge across the full stack, and the ability to make strategic business decisions (exec skills). Is this a missed opportunity, as the value of it seems clear to me? Or is the demand really just not there for this?

I second everything you said above and would like to add that I think another quality security engineer needs to bring is "human skills", by no means I am saying technical knowledge isn't important but human skills are equally important, as part of a job is to interact, teach, persuade people.

As a security professional, most startups understandably outsource security as much as I could so long as the cost met the needs, until I had a product shipped to market and had enough users and reputation to start to care about the risk.

Good luck. I’ll stick to Incident Response and SIEM software.

You make a good point. Let me revise my original statement: There is a huge market for security engineers.

I'm interviewing in SF with ~8 years of product engineering experience and ~2 years of AppSec/SecEng experience. I'm looking at 8 companies that are all willing to pay well for folks to do that work. Typically in range of ~180 - 220k base salary from what I've seen so far.

On the topic of meaningful change: You're absolutely correct in that it's easy for folks in security to find themselves in places where they identify work that needs to happen without receiving support or authority to make it happen. For aspiring technical security folks, there's a few things you can screen for to avoid companies that will do this to you:

1. Does the company have a formal CSIO (Chief Information Security Officer)? If not, move on. CSIOs represent security risks and needs to your executives and board members. Without that, you won't see security work get on anybody's road maps.

2. Does the company have an established security program? If not, do they have a roadmap for making one?

3. What is the size of the technical security team compared to the larger engineering organization? There's no bad ratio here, but the smaller the ratio is, the more critical it is to automate as much as possible.

4. What training programs exist within the larger engineering organization? Do they cover security awareness? Technical security? How well is this program executed? A good training program is critical to reducing new work created for security teams that are typically overloaded to begin with.

There's probably more you can look for here, but I find these questions to be reasonable filters.

I agree with majority of things you mention but wanted to point a couple of things out:

> There is a huge market for security engineers If you have a look on Linkedin for "Application Security Engineer" jobs in London, UK you would find there are not that many, some companies don't even have AppSec Engineers.

> Does the company have an established security program? If not, do they have a roadmap for making one?

For less mature company or company just starting out AppSec could be the force that creates and implements a security roadmap, adopting OWASP SAMM or BSIMM

This matches what I'm seeing as well--smaller companies looking for AppSec often want to bootstrap a new program.

As someone who is pretty much an app sec engineer, I feel like this rings true.

Furthermore, part of me suspects that the tangible business risk of application security flaws isn't felt until after a breach, when its far too late to change things. Even then, sometimes the cost of a breach does not justify the expense of building a robust secure software development life-cycle.

That's a great point. Do you believe that the regularity of significant breaches has cheapened the reputational cost of having experienced such a breach? (Which, in turn, makes it less likely that "a robust secure software development life-cycle" will ever be built.)

I think its worse than cheapening the reputational cost, it has put a concrete ceiling on the financial cost - something like users affected * 2 years of free credit monitoring.

I think you're right, and it's not just security. Nearly any every other application vertical (accessibility, performance, localization, etc.) faces the same struggle compared to adding features. If the org doesn't care about it it doesn't get done. Whoever does do this kind of work needs to think very carefully about how to get their influence to scale.

Absolutely this. Many engineers simply lack security mindedness or secure development training. Similarly, many managers/PMs/etc have gaps here as well. Its important for them to understand how to ask effective questions and prioritize security work accordingly.

I've run into many stupid security mistakes, and continue to do so :(. Even though security can be very hard, as an industry we'd be way better off if more understood at least the basics. Those with interest/expertise can then dive in and find the more tricky things. Those without, can at least understand WHY these things are important.

I think you're right here, and I worry we are overly compartmentalizing security away from engineering. Ideally you should be both. One of the issues I see is that (at least in my experience), security isn't too bad to get up to speed in, if you get the time, but it's pretty time consuming to stay on top of advances.

Cross team working is important, but I think it's also important to try and embed security "people" into the day to day development flow on a regular basis. It ought to help the developers through learning by osmosis.

Definitely think companies need to better signal the demand for deep technical skills in security though - too many seem to actual want generic risk management experience but no particular technical knowledge. To my mind, anyone in a security role should really be able to break their way into a typical corporate internal webapp in a few minutes, just out of their own residual talent and ability to show stakeholders the importance of their profession. It's amazing how quickly security gets a seat at the table when they show how they can pop the self service password reset portal with just a web browser (with appropriate permission of course) and are asking whose password to reset to show it works...

I think it’s even more so a matter of being time-consuming to do correct even if you know what to do... In a microservice architecture. mTLS everywhere, proper PKI, fine-grained access for all users and services, keeping dependencies minimal, vetting updates, securing network, firewall rules, keeping security patches up to date... all this takes time and without strict processes it’s very easy to cut corners.

And what about that PoC that slowly transitioned to bring a production system?

A large part of the blame lies in the culture of pressure on releasing fast and often.

> There is a HUGE demand for application security engineers, or more broadly, security folks with software engineering backgrounds

I wish. On the contrary, in my career I met more people that moved from security to development than the other way around.

In many organizations the priority is on churning out stuff and security comes later.

It is clearly proven by the small percentage of security people VS developers.

As a consequence, developers are paid and praised more. Security people end up fighting an endless uphill battle and sometime seen as "party poopers" by the developers.

As others have mentioned in similar responses, needing security vs supporting security don't always come hand in hand. That said, the technical security jobs are out there, pay well, and aren't receiving much in the way of competition.

To me, that's a fantastic situation as a job seeker. It gives me the luxury of choice. Knowing some companies don't value or support their security teams means that it's something I can screen for ahead of time. If I fail to screen appropriately and end up somewhere that I'm not happy with, I can just start looking again.

Where is this demand you speak of because I have only had jobs building features for Incident Response or being a “DevOps Engineer” that implements secure pipeline, docker, logging, artifact repo scanning...

The need for App Sec is there as far as I’m concerned the demand is non existent... even though most red teams could make them cry.