Was FastCGI a child of a world where we had neither good library use ("import net/http"), nor (much) layering in front of the server (balancers / cdns / cloudflare etc). So it made sense to assume a production-level layer on the box itself was always needed?
I remember the vigorous discussions comparing security of Apache vs IIS etc
But I suspect it's more that CGI was the way things had always been done. They didn't even consider doing a reverse proxy. They asked the question "how do we make CGI faster" and so ended up with FastCGI.
Other developers asked the same question and ended up making mod_php (and friends), embedding the scripting language directly into the web server.
The script would write new html files for new posts and do "fun" (I mean, terrifying) string manipulation on the main index to insert links to posts etc. Sometimes they used comments with metadata to help "parse" pages which would see edits.
These both were, and definitely were not, "the days" :D
Wherever we needed faster interaction, FastCGI did the job and allowed us to interface with anything in the backend including our C programs.
My second biggest customer went out of business. Which left us with a bunch of itty-bitty businesses that I'm just too tired to be chasing after.
The parser ended up being 30 lines of code.