While it is waiting for the fsync to finish it should batch up any incoming WAL writes and then issue the next fsync immediately after the first one finishes, committing the entire batch at once. Then, and only then, it can reply to clients with “transaction complete”.
Some modern database engines now do this, by many older ones don’t because too much of their code assumes one transaction per fsync.
There’s also concurrency issues with writing and fsyncing the same fd: http://oldblog.antirez.com/post/fsync-different-thread-usele...
With automatic batching, trivial transactions can be grouped together so that the bottleneck becomes bandwidth, not an absolute rate.
You get to have your cake and eat it too: There's no additional latency added using automatic batching of transactions because when the I/O queue is empty, the next transaction commits immediately, same as normal. If the disk is already in the middle of an fsync, the next one will have to queue up behind it in the storage subsystem anyway, so the DB engine may as well accumulate more transactions in-memory while it is waiting.
You can have the first transaction wait a bit to see if any other commits can be batched in the same fsync. However that’s off by default as the assumption is you want the transaction to complete as fast as possible.
At least that’s how PostgreSQL implements it.