This was exactly what we tried to avoid with our (rather small) dev team.
It took our infra swe a few months to get MVP version working but he also did other infra related work at the same time. Complexity can change a lot depending on requirements, and ours are probably more stringent that Heroku ever supported. Because of sensitivity of the data we deal with there is now a relatively sophisticated identity management/permissioning/what-can-see-what-data component in how our infra is deployed which probably would not be the case for most companies. We also deploy ML models so there is additional issues with automation around keeping track of reproducability/provenance/ml pipeline regression/drift/deidentification/etc (which now a year later we haven't fully solved either!).