Capacitors do have a different discharge curve than batteries and this requires more sophisticated power management electronics but nothing too difficult. You basically need a power supply/regulator that can start with much less than rated input current and increase current draw as voltage drops to deliver a constant power output.

It takes time for them to discharge, and the rate they discharge can be controlled by the circuit, just like batteries. They just are much less limited in how much power they can handle, so they can discharge the same amount of energy faster. This is a feature, not a bug, as most applications need oversized batteries to handle peak power usage which is substantially higher than average.

You can just slap a resistor in series, which will induce a drop in potential over it. That, in turn, will mean the potential difference across the capacitor is lower, leading to a more shallow exponential decay of the current over the circuit - ie a slower discharge.

And that's just a passive circuit, with an active circuit you have a lot more options.

Note: I studied maths and computing so I could very well be wrong. I would appreciate being set straight by elec eng majors!