It takes time for them to discharge, and the rate they discharge can be controlled by the circuit, just like batteries. They just are much less limited in how much power they can handle, so they can discharge the same amount of energy faster. This is a feature, not a bug, as most applications need oversized batteries to handle peak power usage which is substantially higher than average.