The docs say you should use Vercel's serve or Python HTTP, but since these are just static files you could upload them to S3 for example and serve from there?

Yes, it essentially just builds a collection of static files [1], and you can use whatever tool to serve them.

[1]: https://en.wikipedia.org/wiki/Static_site_generator (ex: Hugo, Jekyll)