My advice is to solve a problem you yourself have.
As a side, the most financially successful side project I started (not calling it a start-up out of respect for your definition) was a vampire themed energy drink - it did not solve a problem for me or anyone else but took advantage of a market created by the entertainment industry. The side project I am most proud of having co-started is a $20/month primary health care network for uninsured/under-insured which was the result of a personal problem - I was uninsured.
This is a subset of "solve a problem someone has".
(Someone = You) is often a sufficient, but not necessary, condition.
(Someone = Someone Else) is an equally sufficient condition, especially for those with subject matter knowledge and industry connections.
Either way, solve a problem.
My advice is just to pick something and start. You'll probably end up someplace completely different, but you need to start to get there.
Telling others to solve a problem they have seems more like an algorithm to generate at least some successes by guaranteeing diversity. This million monkeys approach is a strength of capitalism, but I wonder if it's the method which maximizes expected value for a would be founder. It seems more of a method to guarantee a minimum return for a VC.
This is not one of those cases where founders and investors' interests are opposed.
Option A: 100% chance of receiving $1 billion
Option B: 50% chance of receiving $0; 50% chance of receiving $4 billion
The expected value of A is $1bn. The expected value of B is $2bn. However, most people I know would choose option A, although it has the lower expected value.
I actually think it is easier to enter an existing market, with an existing problem and competitors.
That's not why either were started. Both were started as just cool little projects by guys who wanted the product. They may have rapidly shifted into growth at all costs, but that's not why they were initially created.
I do agree that they weren't created to solve a problem in the way DropBox was, just to scratch an itch of the founders. Which makes growth hacks like Dropbox's referrals much more difficult. Dropbox could survive on people loving the product no matter how many others used it. Snapchat and Instagram value to users craters without a critical mass of users.
Good point about Instagram. You're right, Burbn was started as a startup, it was the photo filter and sharing part that they kept that was Systrom's piece that he just wanted (since he claims to have been square cropping and filtering his photo's for years).
With Snapchat, however, that's not the whole story told by people who knew Speigel at Stanford (my brother was his year). The argument came after they'd already built the app and run around campus showing off how cool their app was. They did, however, very quickly realized that what they built was a company. It's also extremely likely that they had all intended to start a company and that SnapChat immediately became the company once they figured out what they were doing.
Yup. I hope everyone here has read the fantastic Fooled By Randomness.
Which is not to say that Drew Houston isn't awesome - by all accounts he is. But it's easy to give way too much credence to advice from a big success whose specific situation differs from our own. (And everyone's specific situation differs from our own.)
Success is very much luck, but a lot of failure is predictable. There are a large number of startups that simply can't succeed because they have a contrived idea that ticks the boxes on some VC's checklist but that excites no potential customer.
That's basically like saying "buy the winning lottery ticket" in my opinion. There are lots of eyes watching for macrotrends, and lots of people trying to capitalize on that. Identifying the trend is part of success, sure, but that's the easy part - the hard part is separating yourself somehow. Luck isn't the whole reason for that separation, but it's a significant part of it.
I do agree that many founders don't recognize their luck. To their credit though, I think they often emphasize the "work hard, be smart" part instead of saying "I knew I was better than all my competitors because..."
Optimal "near future" distance is the time needed to develop a working version 1.0 (not exactly a prototype) plus some (let's say 20%).
Sounds like a ticket indeed :-)
Many, if not most, founders aren't even aware of these factors, don't appreciate them fully, or won't acknowledge them out of ego. In many cases the advice should be "work hard, be smart, and buy the winning lottery ticket."
My advice is to attempt startups in the center of a confluence of macrotrends; that will tilt the odds more in your favor.