If you know nothing about filesystems, then you're almost certainly limited by the underlying hardware, not the filesystem.

And "performance" is meaningless in this context without specifying what you want to perform. Raw sequential read/write throughput, random metadata operations on directories containing millions of files? Something in between?

    > Are there ways to make a step-change in
    > overall performance somewhere?

Yes, it's called O_DIRECT (or equivalent), and is very useful to those that actually need it.

A general-use filesystem is inherently something that requires you to compromise on performance.