News for hackers. Stuff that matters.
Re:Jeff Dean’s Lecture for YC AI [video] by sputknick on Monday August 07, 2017 @05:32PM (#14951449)
If Tensorflow becomes the default library for Deep learning, is this a good thing or bad thing? Does it help in that all researchers can focus on what's important (the data and results) or does it hurt in that Google now controls an important paradigm for the next generation of computing?
Re:Jeff Dean’s Lecture for YC AI [video] by blueyes on Monday August 07, 2017 @07:53PM (#14952290)
PyTorch is becoming much more popular among researchers, even if TensorFlow use is widespread among data scientists.

Would it be bad if TensorFlow became the default library for deep learning? For companies that aren't Google, yes. Many organizations don't like how Google's controlled Android development.

But I think an important part of Dean's lecture are the slides that talk about making ML/DL expertise obsolete. That's the future, whether you use TensorFlow or not.

Re:Jeff Dean’s Lecture for YC AI [video] by deepnotderp on Monday August 07, 2017 @08:22PM (#14952408)
It's cute, but isn't going to completely obsolete ML/DL expertise. (And believe me, as a prospective supplier of DL compute, I would love for the answer to be "more compute").

To give you an idea of how much compute it took, they spent two million dollars for one run on a relatively toy dataset, CIFAR-10. Imagine how much it would take on an imagenet sized dataset! Can your company afford ~$20 million+ per dataset?

I do think that hyperparameter twiddling aspect might get automated, but believe me, that is much welcomed by the DL research community! I would much rather spend my time on new ideas rather than trying ten different initializations :)

Re:Jeff Dean’s Lecture for YC AI [video] by blueyes on Monday August 07, 2017 @08:34PM (#14952482)
Sure, but that's $2m of today's compute on today's chips. The constraints on chips and cost are moving, and Google is pushing them.
Re:Jeff Dean’s Lecture for YC AI [video] by deepnotderp on Monday August 07, 2017 @11:12PM (#14953721)
Sure, so it becomes ~$2 million for an imagenet scale dataset with TPUs and ~$200K with our chips. Still pretty expensive :)
Re:Jeff Dean’s Lecture for YC AI [video] by londons_explore on Tuesday August 08, 2017 @05:21AM (#14955970)
Android is a very closed opensource project. For example, if you write a pull request to add a feature as an outsider, the chances of it getting into the main project is very low.

Chromium is very open. In many cases, Google has to ask permission from outside contributors to commit code, since the outside contributors are the "boss" of part of the codebase.

Tensorflow is closer to chromiums model. There are a lot of external contributors.

Re:Jeff Dean’s Lecture for YC AI [video] by svara on Tuesday August 08, 2017 @06:44AM (#14956408)
> PyTorch is becoming much more popular among researchers, even if TensorFlow use is widespread among data scientists.

Could you maybe comment on why this is the case? Are there technical reasons for that?

Re:Jeff Dean’s Lecture for YC AI [video] by agibsonccc on Tuesday August 08, 2017 @08:00AM (#14956866)
A big reason is the dynamic computation graphs. Tensorflow fold is supposed to be the response to that, but overall people are finding pytorch to be more flexible.

(Warning: I'm biased. I compete VERY heavily with anything google including cloud and their deep learning framework.)

Re:Jeff Dean’s Lecture for YC AI [video] by halflings on Monday August 07, 2017 @06:05PM (#14951652)
I fail to see what's wrong with this: it's open-source, and anybody can write different libraries, or libraries that use all the low-level work ($$$ paying experienced/skilled engineers) that Google pours into making tensorflow.

Alternatives exist (PyTorch by facebook, theano, caffe, etc.) in any case, and the best solution wins. (in the same way scikit-learn once eclipsed all other ML libraries, but some would still use xgboost which had superior Gradient Boosting Trees).

Re:Jeff Dean’s Lecture for YC AI [video] by ehsankia on Tuesday August 08, 2017 @03:16AM (#14955250)
Absolutely. To me, it's very similar to programming languages. At the end of the day, it doesn't really matter if people use Google's Go, C++, or some other language. What matters is what's produced by it: the actual research and results.

When you see a useful program that helps you be much more productive, you don't really care if it was coded in c++ or python.

The one part Google does care about is being able to sell Cloud TPU, but I don't think that by using this tool you're instantly in under their control.

Re:Jeff Dean’s Lecture for YC AI [video] by davedx on Tuesday August 08, 2017 @06:18AM (#14956239)
Most of the time I agree with this, but some areas (where e.g. Google make Dart but also make Chrome), I think there are risks.
Re:Jeff Dean’s Lecture for YC AI [video] by sillysaurus3 on Monday August 07, 2017 @06:16PM (#14951707)
the best solution wins

Minor quibble, but: not really. The earliest solution wins, e.g. rails. It has to be good enough, but after a certain point "best" stops mattering.

Re:Jeff Dean’s Lecture for YC AI [video] by omot on Monday August 07, 2017 @06:54PM (#14951962)
> The earliest solution wins

Minor minor quibble, but: not really. The earliest solution wins, unless there is a 10x better solution, e.g Google. There were a lot of good enough options, but after a certain benchmark "earliest" stops mattering.

Re:Jeff Dean’s Lecture for YC AI [video] by agibsonccc on Tuesday August 08, 2017 @08:03AM (#14956892)
I personally think this would lead to stagnation in the space. Because of competition, tensorflow is adding better ETL tools, dynamic computation graphs,..

I would be careful about having only 1 company's use cases served. Competition forces parties to innovate.

For example mxnet and cntk or even more recently caffe2 serve different use cases and companies.

Giving you a recent one with our deep learning framework (I compete with all the above frameworks): We compete in the big data niche and recently found a few bottlenecks we missed because people run comparisons.

Re:Jeff Dean’s Lecture for YC AI [video] by li4ick on Monday August 07, 2017 @05:40PM (#14951501)
Researchers use Tensorflow for backend only + Keras for implementation. However, PyTorch and Caffe are much more popular among researchers. I personally enjoy PyTorch a lot.
Re:Jeff Dean’s Lecture for YC AI [video] by niyazpk on Monday August 07, 2017 @05:46PM (#14951535)
>> I personally enjoy PyTorch a lot

Why?

Re:Jeff Dean’s Lecture for YC AI [video] by Hyperbolic on Monday August 07, 2017 @06:14PM (#14951695)
A popular framework for iterative control sequences in deep learning (e.g. Stack LSTM) is Dynet. It's being used more and more in the NLP community.
Re:Jeff Dean’s Lecture for YC AI [video] by harigov on Monday August 07, 2017 @11:43PM (#14953971)
I believe their game plan is more hardware based differentiation rather than software alone. From that perspective, TensorFlow will probably play a similar role to Google's Cloud that DirectX played to Microsoft Windows. As other cloud vendors aren't THAT far behind (notably Microsoft), I don't believe Google will win with that high of a margin, if it wins.
Re:Jeff Dean’s Lecture for YC AI [video] by TuringNYC on Monday August 07, 2017 @06:56PM (#14951970)
I think it is an incredibly positive thing that TensorFlow, Caffe2, Torch and multiple open-source systems are competing actively in the space. These are all active projects, with open-access support facilities. This is all of enormous societal value and we all reap the benefits in numerous, usually invisible, ways.
Re:Jeff Dean’s Lecture for YC AI [video] by tanilama on Monday August 07, 2017 @10:57PM (#14953602)
It won't, however it won't go away either.