I think it's important to internalize that stamps are like a form of currency[0]. Like you stick it on the envelope to pay the shipping fee. There are tax stamps/revenue stamps which are used to pay for things in a way that is hard for the fee collector themselves to steal.

I applied for a visa-related thing a month or so ago. The fee was 6000 yen, so I stuck 2 3000 yen stamps on the application form. I had paid the application fee, the money can't be stolen by the public servant taking the application form (well, not easily, at least, I'm sure there are sleight of hands), and the purchasing of the stamp happens at a trusted third party that could commit fraud but is heavily incentivized not to (on account of me being able to directly cite where I bought it, and the agency knowing who I am).

[0]: Pedants who want to say it isn't or that it's _exactly_ a form of currency: please go away