You can set up filters and forwarding rules. It doesn't require a programmer or special knowledge to do so.
The queue can be viewed by one or many humans.
Messages don't have to be dealt with in-order.
read/unread state, stars, labels, drafts all allow construction of advanced workflows with no specialist knowledge.
Sure, all these things can be done better with special software, but there is a massive benefit to something that all your existing untrained and probably-not-well-paid employees can set up themselves.
> surely HTTP PUT of an XML or JSON document to a user nominated URL
This is the bit that's more tricky than a static site. And then you need to do something with the data.
Often the thing you are replacing is 'fill in this pdf and email to this inbox', so it allows people to improve things for external users without changing their workflow.
"Each form needs an email address to be set for completed forms to be sent to when they’re submitted"
So... erm is an email Inbox needed for integration? - how very 1990!
In addition to email, for use by developers in other UK GOV departments - surely HTTP PUT of an XML or JSON document to a user nominated URL (with provided auth token) would have been trivial to achieve?