There are certainly a few companies in the valley that are doing interesting R&D, but I tend to agree with the article that the vast majority of the crap we hear about on a day-to-day basis is fluff. It may be "Big Data" fluff, but it's still fluff, and popularity doesn't make it less fluffy.
From a technology perspective, cranking out database-backed web pages is a completely uninteresting problem. For that matter, so are most of the common "big data" applications (grinding through log files), or revolutionary companies (where the "revolution" usually boils down to killing a more traditional market...with a database-backed web app). Meanwhile, industrialists in other countries are developing more efficient wind turbines, new electronics manufacturing techniques, battery chemistry with greater capacity, and they're also investing hundreds of billions of dollars into pharmaceutical research. From a technological perspective, it's not even in the same universe.
At the same time in the past month, I've talked with government officials from 2 fairly developed countries, because they are trying to learn what makes Silicon Valley tick. In both cases, they understand that they will never be able to compete with Silicon Valley, but at very least, they want to cross pollinate their technology sector with ours.
I noticed that your profile says that you work at Yelp. I'd respectfully suggest that I haven't seen tons of interesting R&D happen up in San Francisco. There are an inordinate number of social media companies and iPhone app companies up at the tip of the peninsula. But, I tend to hang out in Mt View, Sunnyvale, San Jose and Palo Alto, and there's a much different flavor to the companies down here.
For that matter, so are most of the common "big data" applications (grinding through log files)
What about companies like Palantir? or Cloudera?
I understand Yelp's Log file might be full of shit, but there are countless companies out there which use the same Big Data technology for doing interesting analysis, right from companies like walmart who do transaction analysis to pharmaceuticals like pfizer which use the same technology in developing new drugs. Just because your log file is crap does not means all log files are crap.
Has HN affected your brain or what? You say you received PhD in Computational Biology? I seriously doubt that now.
http://www.cloudera.com/blog/2009/10/analyzing-human-genomes...
Oh btw Yelp is surely one of those useless Web 2.0 companies like posterous. (at least facebook is building social infrastructure and drawing people towards internet ) Last heard you were accused of blackmailing restaurants. How is that working for you.
Before talking crap about Big Data, go out and read about its credible applications.
1)"OSI Level 7" (Social Networking, Web 2.0, Mobile Apps and Widgets companies).
2)"OSI Level 6" (Cloud, Big Data and BI/Analytics).
"OSI Layers 1-5" are more capital intensive and they are heavily underfunded.
My startup somewhere between the "OSI layers 5 and 6" and I fully agree with the people quoted in the article.
While using non-mainstream and exotic programming languages is fun, it has nothing to do with the actual problem domain. One can build "Facebook for cats" using Erlang/OTP and "Twitter for dogs" using Clojure, but they still not solving hard problems, mere a pet entertainment problem.
I have nothing against these languages. I myself using Erlang/OTP for solving problems in Semiconductors industry.
Recently some people refer to it, to explain some concepts. Some even comparing Peninsula in Bay Area to OSI, where is San-Francisco is "OSI Level 7" and Cupertino is "OSI Level 1" ;)
Um...
Zynga...bigger economy than many developing nations
Possibly true. I admire the business acumen of both firms and think they have scaled their services quite effectively. But how productive are they? I mean, I remember seeing stockbrokers and lawyers standing in line at 8am to buy limited edition Beanie Babies because they were considered a smart investment.
You make several good points but I agree with the other posters questioning the degree of value creation taking place.
Computers aren't just for spreadsheets.
- EA CEO John Ritticello, 2009 http://www.gamesindustry.biz/content_page.php?aid=26508
For instance building a faster circuit, a startup whose goal is achieve such a mission has no way of knowing whether it would work, but if it did work there would be tons of people buying.
Lately, people are working on stuff with market risk. Stuff that are market risk tend to be well understood technically, but there is no guarantee that people will buy. For instance a Facebook clone, the chances are slim that people will move from Facebook to this new web application but there is no doubt on whether someone can build it.
Technical Risk generally has the property, if you can build, they would come. Market Risk generally has the property, everybody will build it, more than likely nobody will come[except to one or two].
Which is probably what the author of this article would consider to be part of the problem.
Running Hackers and Founders SV, I think I have a pretty decent picture of what Silicon Valley is like on the ground floor, and as far as I'm concerned it's pretty frigging amazing.
There's tons of small teams doing amazing things with Big Data and real time analytics like Factual (making Open Data easier to manage and use), MixPanel (Real Time analytics for applications). There's a number of companies doing really interesting things with cool languages like Clojure and Erlang, even if they are "silly companies".
There's companies like Justin.TV which is the largest real time video streaming company in the world. Real time video streaming is very, very hard to do at scale.
AirBnB is revolutionizing the hospitality industry, and last I read are the largest provider of rooms in New York.
There are amazingly cool companies working on SSD database appliances, and on databases optimized specifically for SSD's. SSD add on cards for your PCIE graphics slot that give you 6x the throughput of a SATA port. Those companies don't exactly make for exiting reading in Newsweek, so they don't get a lot of press, though.
Yeah, there's a lot of people making Facebook apps. Sure, you can mock twitter because on it's surface it seems silly and trivial, but they have invented an entirely new communication protocol which companies like StockTwits are leveraging to allow individuals to broadcast their recent stock trades. And, not only that, but the article says that Twitter hasn't figured out how to make money yet. In fact, Business Week reports that they were profitable in 2009: http://www.businessweek.com/technology/content/dec2009/tc200...
Sure Zynga makes somewhat whimsical addictive games, but if you stop and crunch the numbers, Farmville's digital economy is probably mlarger than the real world economies of many developing nations.
So, if you'd like to believe that Silicon Valley has lost it's way go right ahead. But, from where I sit, things are just beginning to get interesting. Silicon Valley is just starting to fire on all cylinders since the tech bubble popped in 2000. If you want to know how I'm seeing Silicon Valley: I'm strapping in and getting ready for a great big ride.