There is actually some sort of classification confidence regarding outputs. It's not a binary yes or no for output neurons regardless of what abstraction you've observed in whichever tensorflow or other high level API being used.