Well, using 16bit floats works fine and allows to use more layers, but might be too little.

I'd be interested on a more conservative change using a single byte or maybe 4 bits.