While it is advisable and recommended it isn't absolutely necessary.

A strong math and tech background will help you get started building the nuts and bolts portion of the project. A vivid sense of curiosity will help to pin-point problem that you can attempt to solve.

Many of the APIs might require you to register a company but they don't require specific domain expertise to get started. Once you have an MVP you can pitch it and gain partners that have domain expertise.

Great question and good luck!

Starting a company is not unlike applying for a new job, you should have plenty of relevant experience. If you could easily get a management job in Fintech then you're probably qualified to start your own business. If not, then partner with someone who is qualified.

"Would one need domain expertise to build a startup on <domain>?"

I'm more interested in learning about any domain in which the answer to this is no.

Should I trust my money to someone without experience developing financial applications?

Are you familiar with the incredible amount of regulation in the industry?