Eruditass

Have you actually worked on applying neural networks in the real world?

Calibrating the probabilities of machine learning algorithms is an old problem. By nature of disciminative algorithms and increasing model capacities, yes training typically pushes outputs to one extreme. There is a ton of information still maintained which can be properly calibrated for downstream ingestion, which anyone actually trying to integrate these into actual applications should be doing.

Most recently:

https://arxiv.org/abs/1706.04599 https://arxiv.org/abs/1802.03916