The web is a great leveler, but when it comes to securing funding (in a later stage of your project) it really helps if you are in the neighbourhood of potential investors.
Other than that if you're self funded and you have a project that has a clear business driver making money from the start then it might actually be an advantage to not be in an expensive area.
First of all your retention of employees will be higher, second your cost of living and your salaries and expenses will be lower.
Working in an area like silicon valley you have a much greater chance of working for a startup. But to start one and build it into a business is possible anywhere.
But working for that startup is great experience and you see and learn what it takes to be successful.
I live in Ottawa, Ontario, Canada which has a fairly lively technology community and what seems to be a growing startup community and it always gets me to wondering whether the Silicon Valley thing is just hype or whether it really is the place to be.
The main thing I always consider in this(especially when i consider possibly starting my own startup) is whether it is hype or not. Maybe it becomes a self fulfilling prophecy that Silicon Valley will always be king if all the best talent moves to silicon valley to start or work with new startups.