The mantissa is linear. It's unrelated to how positional number systems work. A floating point value is split into two numbers - the exponent and the mantissa. Normally they are used to represent a final number like:

    x = 2^e * (1 + m)

Where e is the exponent and m is the mantissa (varying linearly from 0 to 1).

But you could have a fully exponential number format:

    x = 2^(m + o)

As pointed out though, it makes addition much more complicated, you can't exactly represent integers, and someone told me it makes quantisation noise worse too. Bad idea.