What I found really interesting was that the reason successful disruptions are "more convenient, simpler and/or cheaper" is not because that's an improvement, but because it enables the disruption to be used by non-consumers... (who lack the skills for a complex product; or the money for an expensive one; or access to an inconvenient one.) They are delighted to have a solution better than what they have now, so it doesn't need to be as good as the incumbents'. Secondly, if it's not good enough to appeal to incumbents' customers, it won't provoke a competitive response.
For example, before the automobile, horse and carriage was a 'solution' to the problem of getting around. In the early days, was a car a much better solution than a horse? Horses were readily available, didn't need a gasoline infrastructure, they even self-replicated so you could get a new one every few years.
I understand I'm playing a bit of devils advocate here, but I often gut stuck in the mode of thinking existing solutions are good enough, then somebody comes along and evolutionizes an industry with what at the early stages might seem to be minor improvements.
Another example would be the early days of sunglasses. If the question was asked 'how are people solving this problem now', the overly simple answer would have been 'they squint a bit'. With that as the answer, would you go and develop sunglasses?
Take your example of sunglasses. A quick search on wikipedia shows that the precursor were glasses with a thin slit in them. It was never squinting, and then BOOM, Oakleys.
There was always a gradual change that is only missed in retrospect because of how standardized and popular the winner is.
Name a few popular products and services, and I bet there have been similar solutions beforehand.
Yep, when you talk about known problems. Still often both the problem and solution (which is just another view of problem) lie just outside current scope of people imagination, and are only found by prospective minds when the way is open (other conditions met).
There was not simply some kind of existing solution for Internet 50 years ago.
There were solutions for this Internet thing 50 years ago, even if it's hardly imaginable today. If the problem you are speaking about is communication we had that 2000 years ago, in form of smoke signals. It sucked and was dog slow but it was not an unknown problem.
think there's a good balance between customer development/research and your own instinct/vision
If founders respond that there aren't really any current solutions, then it usually means that either a) They aren't making something that people really want, or b) They haven't talked to enough users.
If it's a problem people actually have, then they must be coming up with crazy hacks or solutions that are much more tedious/inaccurate/expensive/generally more painful than the one you're coming up with. Very rarely is there simply not some kind of existing solution.