> We probably don't want to display that much precision --- for smaller units, we probably don't want any fractional digits, for larger units like seconds, we probably want two digits of precision maximum.

I really like the way Haskell's Criterion library formats numbers. It always displays four digits total and selects the SI prefix appropriately. I've ported the algorithm to C here, feel free to use it as an inspiration: https://gist.github.com/pkkm/629a66d47ecd16aa89e8b67ba5abd77...