News for hackers. Stuff that matters.
Re:PyTorch 2.0 by hintymad on Friday December 02, 2022 @12:56PM (#33834066)
A big lesson I learned from PyTorch vs other frameworks is that productivity trumps incremental performance improvement. Both Caffe and MXNet marketed themselves for being fast, yet apparently being faster here and here by some percentage simply didn't matter that much. On the other hand, once we make a system work and make it popular, the community will close the performance gap sooner than competitors expect.

Another lesson is probably old but worth repeating: investment and professional polishing matters to open source projects. Meta reportedly had more than 300 (?) people working on PyTorch, helping the community resolve issues, producing tons of high-quality documentation and libraries, and marketing themselves nicely in all kinds of conferences and media.

Re:PyTorch 2.0 by SleekEagle on Friday December 02, 2022 @02:18PM (#33835377)
Exactly, especially in the age of ridiculously rapid development that we have found ourselves in over the past few years. This is exactly why TensorFlow is dying
Re:PyTorch 2.0 by la_fayette on Friday December 02, 2022 @02:57PM (#33835984)
:) it still is impossible to bring a cnn rnn network in pytorch to mobile, which works fine with tflite...
Re:PyTorch 2.0 by levesque on Friday December 02, 2022 @05:14PM (#33837681)
I don't think that's a problem for the vast majority of pytorch users
Re:PyTorch 2.0 by antixk on Friday December 02, 2022 @08:31PM (#33839335)
Not sure if you have heard of Pytorch Mobile but it is very possible[0]

[0] https://pytorch.org/tutorials/beginner/deeplabv3_on_ios.html

Re:PyTorch 2.0 by perturbation on Friday December 02, 2022 @11:34PM (#33840634)
The big thing that PyTorch Mobile is lacking compared to TF Lite is on-device accelerator support (GPU/DSP/etc.) (there's experimental support for NNAPI https://pytorch.org/tutorials/prototype/nnapi_mobilenetv2.ht..., but this is a hack).
Re:PyTorch 2.0 by amitport on Saturday December 03, 2022 @06:29AM (#33842684)
I do not believe TensorFlow is anywhere close to dying.
Re:PyTorch 2.0 by wging on Friday December 02, 2022 @03:20PM (#33836330)
> Meta reportedly had more than 300 (?) people working on PyTorch,

How much did this change after the big Meta layoffs? I think I know people who are no longer there, but I haven't talked to them about it yet.

Re:PyTorch 2.0 by not2b on Friday December 02, 2022 @03:33PM (#33836492)
Because Meta open sourced it, and because PyTorch caught on, hopefully some of those laid off people can continue to work on it and also market themselves as PyTorch experts.
Re:PyTorch 2.0 by PartiallyTyped on Friday December 02, 2022 @04:46PM (#33837372)
NB: PyTorch is now under the Linux Foundation.

https://news.ycombinator.com/item?id=32810976

Re:PyTorch 2.0 by vikinghckr on Friday December 02, 2022 @05:28PM (#33837821)
What I really find interesting here is that PyTorch, a library maintained by Facebook, is winning the marketshare and mindshare due to clean API, whereas Tensorflow, maintained by Google, is losing due to inferior API. In general, Google as a company emphasizes code quality and best practices far more than Facebook. But the story was reversed here.
Re:PyTorch 2.0 by miohtama on Friday December 02, 2022 @05:57PM (#33838113)
An interesting question. Maybe Google lacks the culture to work with external developers (think Android) while Facebook has some of it.
Re:PyTorch 2.0 by roenxi on Friday December 02, 2022 @08:04PM (#33839150)
Google's stereotype is the company that can handle and manage complexity - things like Kubernetes, indexing the internet, etc. They are clumsy at persuading people to use their products and have a patchy history of launching platforms that people want to use. Google+ and Google cloud vs AWS spring to mind, Kubernetes is a good platform but challenging to learn. Chrome is an unusual aberration where they did a great job. Android is maybe also a strong counterexample, not sure what the state of play was there.

But whatever their code quality may be, luring in devs to their API has seen hits and misses.

Re:PyTorch 2.0 by wartijn_ on Saturday December 03, 2022 @02:12AM (#33841424)
They did a pretty good job in persuading people to use search, adsense, YouTube, Gmail, Maps and a bunch of other products. Singling out a few couple of the less successful ones to claim a trillion dollar company is “clumsy at persuading people to use their products” seems like a pretty bad take.
Re:PyTorch 2.0 by roenxi on Saturday December 03, 2022 @05:28AM (#33842387)
You'll note that all the examples you're citing were released more than a decade ago (search wasn't even this century!). And YouTube was an acquisition. They were basically the work of a different Google to the one that is walking around now.

The stereotype of Google back then was very different. People would quote things like "don't be evil".

Re:PyTorch 2.0 by fsociety on Tuesday December 06, 2022 @09:44AM (#33880699)
The only products you listed which weren’t acquisitions are search and gmail. Search is exactly the problem Google was made to handle and Gmail.. well I don’t love gmail but it is a good enough product I guess?
Re:PyTorch 2.0 by ShamelessC on Friday December 02, 2022 @08:28PM (#33839316)
Google historically has a culture of writing libraries with other Google employees as the target audience.
Re:PyTorch 2.0 by pjmlp on Saturday December 03, 2022 @01:26AM (#33841206)
The way Android NDK and AGK experience works, I always have the feeling it is maintained as a 20% project and none of the developers has ever used anything from the competition.
Re:PyTorch 2.0 by aschleck on Friday December 02, 2022 @10:01PM (#33840095)
It's worth noting that Google is also working on JAX these days and that's picking up steam, and one reason is because its API is super clean.
Re:PyTorch 2.0 by chem83 on Friday December 02, 2022 @10:23PM (#33840225)
And ML compilers in the form of IREE/MLIR, which if successful in delivering hw and framework optionality, should make ML frameworks a far less consequential decision.
Re:PyTorch 2.0 by rawrawrawrr on Saturday December 03, 2022 @02:05AM (#33841382)
Not really, because people will still use the dominant framework because that's where all the research, examples, libraries are.
Re:PyTorch 2.0 by ldjkfkdsjnv on Friday December 02, 2022 @10:10PM (#33840157)
The same thing is true of react vs angular. Angular is kind of garbage, whereas react is elegant
Re:PyTorch 2.0 by rawrawrawrr on Saturday December 03, 2022 @02:07AM (#33841392)
Google has a culture of encouraging their developers to build complex things, which is important for promotion.
Re:PyTorch 2.0 by niels_bom on Saturday December 03, 2022 @04:46AM (#33842190)
Couldn’t these organizations be big enough to have different technical cultures from team-to-team or product-to-product? If that’s the case comparing FAANG to each other is like comparing Asia to Europe and then taking North-Korea and Portugal as examples.
Re:PyTorch 2.0 by solarkraft on Sunday December 04, 2022 @02:24PM (#33856793)
> In general, Google as a company emphasizes code quality and best practices far more

That doesn't automatically have to mean better DX. For example the API can be clean, but too low level to conveniently accomplish typical cases (I'm not familiar with this example).

Re:PyTorch 2.0 by t-vi on Wednesday December 07, 2022 @08:51AM (#33894302)
> Another lesson is probably old but worth repeating: investment and professional polishing matters to open source projects

While I'm not going to disagree that it matters, to my mind the main thing about early (0.x, 1.x) PyTorch's success was a clear vision (most visible to me from Soumith and Adam) that put the targeted users first (as in your productivity comment), awesome execution on it, and a creating a community where I, as an outside contributor with modes skill and no AI track record, and likely many others, felt welcome.

Now, the many great people employed by Meta, but also other players like MS, NVidia, AMD, Intel, ... to work on PyTorch enabled all the things that make PyTorch 2.0 nicer and more broadly applicable than 1.0. (And to my mind it's quite a non-trivial accomplishment to enable other corporate players to join the party.)