1. Hiring managers on the defensive side of infosec tend to have an "us vs. them" attitude towards attackers. The people in the trenches in this field are young and open-minded, but management can tend towards grey-haired and conservative.
2. Infosec is closely tied to risk management. Nobody is allocating enough resources to security to properly mitigate threats, and everyone's acutely aware of that. Lots of large, smart firms are going to avoid hiring people with criminal backgrounds simply to avoid the possibility of an own-goal.
3. Relatedly: even firms that spent nosebleed amounts of capital on security are going to have high-profile security events, and when that happens, the rest of the organization is going to look to the infosec team for someone to blame. The backgrounds of everyone on the team will be relevant in those finger-pointing exercises.
4. Finally, and as I've said elsewhere: the offensive side of security is dominated by contractors, and contracting clients tend to background check as a matter of course.
You can nurture an interest with software security as a developer, by being the developer on your team that knows the most about software security; you can keep doing the security stuff you like now without necessarily holding out for a role with "security" in the title.