It would be useful beyond some custom git-based solution because if what you want is just a distributed ledger with safety & auditing built-in you can get that off the shelf by leveraging e.g. the Bitcoin codebase. If you're doing that with Git you need to write a bunch of custom software around it to arrive at the same result.
The bitcoin codebase doesn't provide anything as good as you describe.
For example, let's say you wanted to find, in the ledger's history, a transfer of $20 from $a to $b. With bitcoin, that requires writing custom software. Git has built in tools to find changes with certain contents, signed by certain keys, etc.
If you want to argue about what has more mature tooling for the banking usecase, you'll find that literally anything that isn't a blockchain, be it git or a sql database, has more mature auditing tooling.
That's the benefit to financial institutions, and the threat to SWIFT. The technology itself is nearly irrelevant. From a game theory perspective, what matters is whether their R&D is plausible enough that they can use the tech as leverage to negotiate lower fees.
Also known as cutting out the middleman.
How does a distributed ledger remove/lower settlement costs?
Tell me how this theoretical private blockchain would be better, in any way, than a git repository where each bank signs commits, commits them, and pushes those commits to all participating banks.