News for hackers. Stuff that matters.
Re:Goodbye Academia by pyrrhotech on Monday March 03, 2014 @04:12PM (#7336517)
It's funny to me when people call companies like Twitter "tech companies". What technology has twitter invented? There are open source clones with 70% of the features in < 1k LoC. Bootstrap? give me a break.

It's time we got rid of this misnomer. Just because part of your company revolves around a piece of software you wrote doesn't mean you are a tech company. Writing software is not necessarily creating technology. Otherwise anyone who's written their own html site has just progressed technology.

Re:Goodbye Academia by simonw on Monday March 03, 2014 @04:29PM (#7336624)
"What technology has twitter invented?"

http://twitter.github.io/ currently lists 101 public repositories. Highlights include a SPDY library for iOS, Zipkin (a distributed tracing system), flockdb (a distributed, fault-tolerant graph database), Finagle (their own highly concurrent RPC system)... a TON of stuff.

Re:Goodbye Academia by hatred on Monday March 03, 2014 @05:15PM (#7336912)
No offence meant but that is like saying "Kia is also a car, does the same work. Why the hell even BMW exists"

May be BMW is doing things on the back that make it a better choice. Same is the difference between twitter and its thousand clones.

Re:Goodbye Academia by dajohnson89 on Tuesday March 04, 2014 @03:05PM (#7342711)
Do you have any idea the amount of engineering effort required to handle the scale of Twitter's service?
Re:Goodbye Academia by scrollaway on Monday March 03, 2014 @04:23PM (#7336575)
Companies like Twitter, Facebook, etc, regardless of what technology they invent that is of use for the public, often healthily contribute to the open source ecosystem. Twitter has contributed a lot to the Scala ecosystem, and Facebook to the PHP pseucosystem.

They also fund/maintain OS project they themselves use. Facebook, through their FriendFeed acquisition, obtained Tornado [1] and have maintained it very well since. I use Tornado every day.

I get your point but your anger is misplaced if you think the large "SV tech companies" don't "invent". You want tech companies that don't contribute anything to mankind? Look at purely closed source hardware companies.

[1] https://github.com/facebook/tornado

Re:Goodbye Academia by silverlake on Monday March 03, 2014 @04:18PM (#7336552)
You can write a superficial clone of any web app. But can you handle 8000 tweets/sec in real-time? The innovation is in the back-end.
Re:Goodbye Academia by pyrrhotech on Monday March 03, 2014 @04:22PM (#7336571)
It's been done notoriously badly at Twitter, no offense if you are an employee. Sure it's gotten better since they switched to the JVM, but still scaling to that size has been done by hundreds of companies now. Once there are textbooks out describing the techniques, you are simply implementing known methodologies with perhaps a few customizations and tweaks. I don't really see much innovation there TBH.
Re:Goodbye Academia by silverlake on Monday March 03, 2014 @04:42PM (#7336703)
I'm neither an employee nor a twitter user. But I've built phone networks and appreciate the absurd complexity of very large systems. If you believe you can build twitter, you're either a brilliant programmer or ignorant of the real challenges. Look at Twitter's code on github and tell me that's just a simple tweak on a known methodology.
Re:Goodbye Academia by AllTheThings on Monday March 03, 2014 @04:59PM (#7336817)
Twitter, Facebook, and LinkedIn (along with other big tech companies) all produce a lot of academic research that documents the architectures and protocols that they use, along with their performance. These companies innovate, and the material from their research often makes it into these "textbooks ... describing the techniques".
Re:Goodbye Academia by cnlwsu on Tuesday March 04, 2014 @11:50AM (#7341358)
Lets not trivialize a truly difficult problem just cause mongodb has some huge numbers on their marketing literature. Its naive to believe that scaling to the level of twitter is easy.

Sometimes you have to be innovative just to implement simple ideas in those text books or to make it actually efficient. Its easy to use type 1 UUIDs as ids but if its not practical to store 128bit ids for every value or you need to pack it into 64bits or need better ordering guarantees. you build something like snowflake.

Re:Goodbye Academia by tricolon on Monday March 03, 2014 @04:25PM (#7336601)
Surely you would consider at least some of Twitter's open source projects "creating technology": https://github.com/twitter
Re:Goodbye Academia by dredmorbius on Monday March 03, 2014 @05:18PM (#7336939)
As much as I find many of the past decade of Web / tech companies underwhelming, innovation can come from unusual places. Duff's Device, a method for unrolling loops in C for better performance, was created at Lucasfilm.

The challenges of scaling to very high levels has presented engineering challenges and solutions. I'm not sold that they're net beneficial in the greater order of things, but to say that there's been no significant innovation would be false.

Re:Goodbye Academia by abrichr on Monday March 03, 2014 @04:26PM (#7336603)
Twitter is one of the main contributors to Storm:

http://storm.incubator.apache.org/