It's the same urge that makes us all think we're better drivers than average, better looking than average, etc. It's the urge that make people move to Silicon Valley/Hollywood/etc., because everyone's going to "make it" even though statistically, that isn't possible. It's why every parent with the means pays through the nose to get their kids into Harvard, why SUV sales are high even though most people don't off-road, and why people buy technical outerwear for climbing, etc. if they never go near a mountain.
Everyone has a constructed image of themselves and we're lying if we say we don't try to burnish it. I think using all this tech is an expression of the desire for status. The problem is, it really can be a time-consuming, frustrating, customer-data-losing, unsupportable, waste of resources. That's bad for business, bad for your health (spending time fixing shit rather than eating well or going to the gym), and it can make your friends and family irritated with you, too.
I'm building a product right now on the most boring stack possible: python, postgres, pyramid. I've been there/done that on the whole sharded-MongoDB-Redis-Memcache-900 machines-AWS-Docker-startup-ops nightmare-use all the things (we actually used every single thing in that list at my last job). It was unreliable, a huge pain in the ass, a little bit fun, but ultimately quite counterproductive. These days, my bets are on things like the JVM, the .NET CLR, and Postgres. As Knish from Rounders would say, "My kids eat".
Another thing to keep in mind: a lot of this is resume padding. Yes, this is completely unprofessional, reckless behavior, but it happens, many recruiters are keyword-driven, and as interviewers many of us are impressed with a keyword-laden resume. We do this to ourselves as an industry in how we hire, who we let speak at events, and who we praise. It's nobody's fault but our own.
My sibling post is on the right lines - a lot of people in tech are chasing a delusional self-image - though I don't think that's true of everyone. People can make the effort to be self-aware and honest with themselves. I think the rapid expansion of tech in the last few years attracted many, many youngsters, hucksters, delusional types, and so on, and people with maturity and experience became a minority.
To be frank, the tech community has really been leaving a sour taste in my mouth the last few years. Everyone is anxious to bandwagon on these buzzwords that provide little to no benefit or are applicable only to a small segment of the tech population.
In most cases, even a superficial understanding of the problem space should make it obvious that $This_Weeks_Sexy_Solution is a very bad fit. Do you need a server that persists data and is individually addressable? Then why are you using Kubernetes and Docker, which are still struggling to figure out these very basic things? k8s and Docker have very specific uses, but unless you're Google, they're probably not the right fit for your production environment right now.
This phenomenon seemed to hit a critical mass with document databases and single-page apps, and it continues to iterate with every open-source release that comes out of Google or Facebook. Since Google released TensorFlow last year and the compounded hype of [much worse than advertised attempts at] conversational speech recognition in Siri and Alexa, the "machine learning" bandwagon is starting to try to edge into the spotlight, and it sounds like it's already a mandatory part of any VC pitch.
Fortunately, machine learning is pretty hard and you get into hairy math practically right away, so I don't think the legs on this one will last as long. But we'll see. We'll at least have a lot of faux-ML going around and a lot of people making spurious claims on their resumes.
The economic collisions that make Silicon Valley and the tech industry in general a hive for inexperienced and insecure youth are bearing some really interesting effects this way. How can a company that's not "blown about by every wind of [tech fad]" fully exploit its relative sanity for competitive advantage?
[0] https://news.ycombinator.com/item?id=13572415 ; my reply downthread at https://news.ycombinator.com/item?id=13573978