Depending on what you mean by the "late '90s" I don't think it was quite that bad, but the budget was still up there. Apache was good enough for most if not all purposes by then, if you really knew what you were doing and your database people collaborated really well you could save 1/2 by using IBM's DB2 (very little independent documentation and lock vs. MVCC concurrency). Of course assuming your VCs bought off on all that.

Another problem is elasticity; do your architecture correctly and AWS will allow you to quickly scale. Otherwise you tend to buy too much or too little hardware, either of which can kill or damage you.

By then the web was quite useful for finding technical answers; obviously not as good as today, but still very useful.