Yes, valuations would shoot to infinity if investors believed that interest rates would reach 0% and then stay there forever. Finite valuations are based on the idea that, at some point in the future, interest rates will be above 0%.

Also, although interest rates for the national governments of some countries have reached 0%, or even negative numbers, commercial interest rates are still slightly positive.