If you were to go down deeper, a server is just an electrical machine which shuffles eletrons around. So, you could say "there's no point in talking about 'servers', if we're just using transistors when you really think about it".
But it would convey no useful information if someone asked you "what are you using to run your service?" and you replied "well, I just move electrons around", would it?
As far as managing goes, most of the server are not managed by you anyway. But yeah i get it, everyone likes it so...
In that scenario, not having to even consider if the error you're getting is because your server is restarting/out-of-memory/needing-update/broken-by-a-coworker like 99% of the time is as close to "serverless" as it gets, in terms of day-to-day work activities and worries.
So it does allocate virtual infrastructure?