News for hackers. Stuff that matters.
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by jiayq84 on Tuesday April 18, 2017 @02:42PM (#14141613)
Hey guys - Yangqing here, worked on Caffe and Caffe2 over the years, super excited to have continuous contribution to the OSS community. I am more than happy to answer questions if you are curious.
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by IshKebab on Tuesday April 18, 2017 @03:27PM (#14142060)
How difficult is it to set up custom recurrent networks? The only system I've ever seen that handles this well is CNTK - you can just say `x = PreviousValue(y)`.

I've tried to work out how to do similar things in Torch and Tensorflow but all they really offer is pre-packaged layers like LSTM. If you want to make your own it's difficult, undocumented and not at all ergonomic.

How does Caffe2 compare?

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by spangry on Tuesday April 18, 2017 @08:04PM (#14144268)
I am a complete machine learning noob, so this could just be my lack of skill. Having looked at most of the popular ML frameworks, none of them seem to provide an easy, functional way to implement a custom 'neuron'. That is:

  function (someinput):
      output = do_stuff
      return output
Which then gets passed on to the next layer. Again, it's quite possible I've just missed something, or there is some inherent ML limitation that makes this impossible. But if such a thing is possible, it would be a rather awesome feature from my POV.
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by Q6T46nT668w6i3m on Tuesday April 18, 2017 @09:58PM (#14144873)
Many do. For example, the `Lambda` Layer from Keras:

https://keras.io/layers/core/#lambda

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by spangry on Wednesday April 19, 2017 @06:47AM (#14146817)
Yeah tried that one a while ago but didn't have much success. I'm a bit of a dummy when it comes to ML. That said, it looks like they've updated the documentation or something, so might be time to give it another whirl. Thanks for the reply.
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by IshKebab on Wednesday April 19, 2017 @08:45AM (#14147301)
This is another area CNTK's 'BrainScript' excels. You can pretty much express it as normal maths:

https://github.com/Microsoft/CNTK/wiki/BrainScript-Activatio...

List of functions:

https://github.com/Microsoft/CNTK/wiki/BrainScript-Full-Func...

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by sirfz on Wednesday April 19, 2017 @07:00AM (#14146875)
tensorflow's py_func allows you to do that, you can also write your own C++ operators, their API interface for that is pretty nice. In fact, you should be able to pretty much write any op you need by combining existing ops (maybe not as efficient as you'd like but yields the results you want).
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by spangry on Thursday April 20, 2017 @08:04AM (#14155878)
Sorry I'm late to reply. This looks perfect for what I have in mind! Thanks!
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by dzhulgakov on Tuesday April 18, 2017 @03:35PM (#14142138)
The recurrent network work is under way - we are revisiting the designs to consciously balance between performance and API niceness. We'll share more tech details in the upcoming days. Right now, this is an example RNN:

https://github.com/caffe2/caffe2/blob/master/caffe2/python/e...

We use it for machine translation so the perf is nice.

If you have design feedbacks please let us know - we'll be really grateful.

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by iraphael on Wednesday April 19, 2017 @10:09AM (#14147790)
I know one of the biggest struggles of training RNNs in TensorFlow, for instance, is the fact that the graph is static. Other frameworks, like PyTorch, have dynamic graphs.

Will Caffe2 support dynamic graphs (perhaps down the line)? What is your reasoning behind whichever decision?

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by drej on Tuesday April 18, 2017 @02:54PM (#14141748)
A super inexperienced observer here, so just a very basic question - how does Caffe(2) differ from TF/Theano/Torch etc.? What are the obvious upsides and potential downsides? A tweet-sized answer will do, I'm just curious as to what the high level differences are.

Cheers and congrats on the release!

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by smhx on Tuesday April 18, 2017 @02:56PM (#14141768)
http://caffe2.ai/docs/caffe-migration.html#null__how-is-caff...

> Caffe2 is built to excel at mobile and at large scale deployments. While it is new in Caffe2 to support multi-GPU, bringing Torch and Caffe2 together with the same level of GPU support, Caffe2 is built to excel at utilizing both multiple GPUs on a single-host and multiple hosts with GPUs. PyTorch is great for research, experimentation and trying out exotic neural networks, while Caffe2 is headed towards supporting more industrial-strength applications with a heavy focus on mobile. This is not to say that PyTorch doesn’t do mobile or doesn’t scale or that you can’t use Caffe2 with some awesome new paradigm of neural network, we’re just highlighting some of the current characteristics and directions for these two projects. We plan to have plenty of interoperability and methods of converting back and forth so you can experience the best of both worlds.

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by mtw on Tuesday April 18, 2017 @04:22PM (#14142541)
does it mean PyTorch or Tensorflow are better for research, experimentation and Caffe2 was more designed for production?

Or it actually means Caffe2 was designed for both research and production

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by jiayq84 on Tuesday April 18, 2017 @08:28PM (#14144394)
PyTorch definitely makes experimentation much better. For example, if you want to train some system that is highly dynamic (reinforcement learning, for example), you might want to use a real scripting language which is Python, and PyTorch makes that really sweet.

Sometimes the line gets a bit blurred - for research that are focusing on relatively fixed patterns, such as Mask RCNN, both PyTorch and caffe2 are working great. In fact, Mask RCNN is trained in Caffe2, and that also makes things much easy when we put it on mobile - what our CTO Mike Schroepfer showed in his keynote is a Mask RCNN model trained and then deployed onto mobile with Caffe2.

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by binaryzeitgeist on Wednesday April 19, 2017 @05:43AM (#14146575)
When are you planning to release the Mask-RCNN code?

I'm trying to implement the RoIAlign layer in Tensorflow and I've a few doubts and having the author's code would definitely help in implementing it.

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by anand_MSFT on Tuesday April 18, 2017 @11:28PM (#14145242)
We launched a Data Science VM supporting caffe2 - https://blogs.technet.microsoft.com/machinelearning/2017/04/...
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by danielmorozoff on Tuesday April 18, 2017 @03:01PM (#14141822)
Would also love some information on performance. How does Caffe2 stack up against TF1.0+? When is MKL support planned for ?
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by jiayq84 on Tuesday April 18, 2017 @03:06PM (#14141878)
RE the performance - many frameworks nowadays enjoy the high-performance libraries such as CuDNN for optimized runtime on different platforms. For example, we've been collaborating with NVidia on optimized distributed training on GPUs, and you can check out more details here: https://blogs.nvidia.com/blog/2017/04/18/caffe2/
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by danielmorozoff on Tuesday April 18, 2017 @03:13PM (#14141952)
Thanks for the link. Are there any public specifics comparing distributed tf to Caffe2's distributed implementation?
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by jiayq84 on Tuesday April 18, 2017 @06:07PM (#14143432)
I am not aware of that right now. Internally we measure the performance by looking at the theoretical peak possible (like the scaling efficiency when you use distributed training). C2 has been doing pretty well; if you find performance degradations, send an issue to us and we are usually very good to figure out the perf.
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by jiayq84 on Tuesday April 18, 2017 @03:04PM (#14141848)
MKL is going to be merged very soon - a full MKL pull request is here if you'd like to track it:

https://github.com/caffe2/caffe2/pull/270

And thanks so much for the awesome work from Intel!

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by mto on Thursday April 27, 2017 @02:14PM (#14213603)
Great work. I would like to know "how beta" the Windows version is. Is there a small library for inference that we can use integrated in an application?

I am currently exporting models from an existing Theano system running on an a Linux server. I'm currently running inference using tiny-dnn and an alternative direct implementation using Eigen (which is much faster) but in the long run it would be nice to have stuff like quantization and GPU available. Thanks

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by laughingman2 on Wednesday April 19, 2017 @01:55PM (#14150022)
Why do you people at Facebook use a patents clause and destroy integrity of open source license of BSD?

Better create a proprietary license which allow you to protect your predatory patent claims instead of acting benevolent and releasing with this kind of corporate malarkey.

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by cs702 on Tuesday April 18, 2017 @03:35PM (#14142141)
Yangqing, thank you. This is awesome.

Quick question: why did you develop and release this as a completely separate codebase from the original Caffe you created at UC Berkeley?

Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by jiayq84 on Tuesday April 18, 2017 @06:03PM (#14143399)
Ah, it is because we do not want to create too much chaos for people to migrate between codebases. Evan and I had played both scenarios and decided that it is cleaner to put the code in separate codebases. A lot of the runtime code are shared, and we are working on the migration tools - such as model converters - that helps migration. If there are bugs in these tools, shoot us an issue on github and we'll fix it.
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by cs702 on Tuesday April 18, 2017 @07:06PM (#14143847)
Makes sense. Will do. Thank you for the quick answer!
Re:Caffe2: Open Source Cross-Platform Machine Learning Tools by nzjrs on Wednesday April 19, 2017 @02:36AM (#14145929)
Will there be any project management or stewardship of this project,l? unlike caffe1 where every paper, project, or hypothesis yielded a mutually incompatible fork on github?