Austrian school of economics believes that occasional recessions are useful because they destroy bad ideas (not exclusively bad ideas, but many bad ideas) and therefore free up resources for new ideas. Some of which will be good, some bad.

For a huge example: Madoff was only discovered because of the 2008 crisis. SEC had plenty of evidence against him for years before that, including the fact he did not actually trade! The guy didn't even trade securities and posted consistent double digits returns! The SEC did not investigate. Only when clients started asking for funds has the scheme collapsed.

I that is the SEC for you. At one if the firms I worked at we had an algorithmic trader who did hundreds of thousands of naked shorts (selling short a stock he didn’t own). He always canceled the order, bought shares, or swapped internally but was still a no-no, and he would do this thousands of times a day. He got caught and we put together a report of his activity that required a hand truck to deliver. The SEC took one look at that and made the firm promise to increase their internal oversight and that was it. They didn’t even open any of the boxes.

Naked shorts are still legal if you're a market maker, for example. They're not entirely bad, it's sort of like credit - it's only bad when you fail to deliver. If you do deliver then it's great as it lubricates the system. See "Regulation SHO":

https://www.sec.gov/investor/pubs/regsho.htm

Not sure if this was the case at your company, i.e. if they had the permission to do so. If not then I don't know why they didn't prosecute that trader. Most organizations cannot do naked shorts and have to pre-borrow securities.

    occasional recessions are useful because they 
    destroy bad ideas (not exclusively bad ideas, 
    but many bad ideas) and therefore free up resources 
    for new ideas. Some of which will be good, some bad.

I feel this was absolutely true for the world of internet applications.

The stuff built in the early dot-com era was typically garbage from a technical standpoint. Heaps and heaps and heaps of spaghetti code written in PHP, Perl, and ASP Classic. You could theoretically write maintainable code in those technologies if you were truly dedicated but this was decidedly not the norm.

Think of literally any basic best practice for writing either the server or client portions of a web application that we might follow in 2019. In, say, 1997 those best practices hadn't been invented yet and even if you traveled back to 1997 in a time machine you couldn't implement some of them since the technology hadn't been invented yet.

From a technical perspective, roughly 2002-2006 was when things started to get really good from a technical standpoint.

- A lot of untalented folks (people calling themselves "developers" but with no real coding ability) were filtered out of the industry.

- A lot of lessons had been learned about how to build this stuff.

- More and more people were getting high-speed internet access.

- The use of AJAX to build responsive client-side applications became a thing, thanks largely to...

- ...Google, who released two absolutely world-shaking web applications in Gmail and Google Maps. These became something of a guiding light in a number of ways. Both from a technical showcase of what could be accomplished with web standards, and a design perspective... it showed a lot of pointy-haired bosses that what people wanted was minimal design and maximum usefulness, not a flashy multimedia experience.

- Other "Web 2.0" tech like RSS matured and enabled LOTS of cool demos (sadly we've moved away from this...)

- Pressure from Firefox forced Microsoft to finally embrace some more web standards and we edged away from the brink of a Microsoft-ruled internet

Dunno. Practices in 1997 and 2002 didn't differ in very appreciable degree, just that piles of crap C++ became piles of crap Java. AJAX was made possible not by Google but by Microsoft's XHR and IE5. RSS never took of in any appreciable commercial way…

I would say the bust had no substantial impact on technology side of thing. Perhaps the pricing became bit more modest, and sure, lots of careers were "filtered out".

>I would say the bust had no substantial impact on technology side of thing.

I think that it did, because it focused development efforts of companies that remained in business. Many failed dot-coms should have failed before the bust, but they kept acting like working businesses as long as they had capital available. They diluted markets, absorbed development talent by offering equity and large salaries, and in general distorted the whole industry.

After the bust, the technologies and idea that got attention/effort/built up were the ones with a solid foundation, not the pipe dreams.

Ajax was made technically possible by Microsoft XHR and IE5. It took Gmail and gmaps to bring this to wide attention.

Yes. I remember people (including me) being a bit amazed by the web version of Outlook that first made use of it. Technically MS introduced this functionality in IE5 in 1999, as an ActiveX control.

It was a Microsoft-only thing for a while though, until Mozilla/Safari/Opera implemented their own versions. That took a few years IIRC.

At any rate, Gmail was what made the world really take notice of what AJAX could do.

For the record there where a lot of people that where doing the same thing before the XHR object, they where just using an iframe and polling to get the data which was usually an HTML snippet or XML. They would construct the url for data with JS in the main frame, change the url of the hidden frame to point to a different CGI script, get the data and read it into JS variables. It was just that none of them got the exposure of Gmail or Outlook web. Now in saying that iFrame polling was a huge hack and a major PITA but it worked. IIRC one of the major CGI/Perl chat apps used this hack to update the web chat window.

One thing I will add too, is if you do go back to the enterprise or at least out of tech, make sure you don't allow yourself to stagnate or miss the upswing. I saw that happen to friends who got salary stuck making half what they would had they joined the broader market during the recovery. Enterprises can rescue you in some instances, but they are not usually great for salary growth outside of a few key places and people.

So I'd only caution people to pay close attention the whole time so you can know when to jump back into the market and to capitalize on that next salary increase.