You've mostly heard correctly. With the difference, that most tech people who are officially hired as techs by banks aren't very good. Good tech people are usually recruited by rebellious front and mid office desks.

Depends on the system. If it's a brand new one there will usually be a set of global firm standards that should be followe (i.e. Oracle, Linux, Java or whatever) but you can follow a process to get an exception. It's not great but as banks have 3,000+ systems of various types they need to keep some control of the technology stacks in use.

IT is still back office, but in the last couple of jobs where I have dealt with the Front Office directly no-one laughs at you. They can't do their job without technology.