News for hackers. Stuff that matters.

A comment on Swift: Google’s Bet on Differentiable Programming.

Re:Swift: Google’s Bet on Differentiable Programming by scottlocklin on Thursday April 09, 2020 @09:38AM (#22822394)
I had to google "differentiable programming" to see if they were really touting automatic differentiation as some futuristic language feature. Honestly the only real response to this should be :trollface:

I've been doing software development in data science, large scale optimization and machine learning for over 15 years... I've needed automatic differentiation in my language .... exactly never. I mean, most of the languages I use regularly are capable of it, and it is a neat trick; it's just not that useful.

The best part of this article is Yann and Soumith twittering they need Lush back (and not because of automatic differentiation). I agree; it's still my all time favorite programming language, and I don't even fool around with Deep Learning. https://twitter.com/jeremyphoward/status/1097799892167122944

[ Parent | on HN ]
Re:Swift: Google’s Bet on Differentiable Programming by abra1010 on Thursday April 09, 2020 @09:40AM (#22822422)
If you're doing "deep learning" using frameworks like Tensorflow, PyTorch or JAX you're using autodiff all the time.
[ Parent | on HN ]
Re:Swift: Google’s Bet on Differentiable Programming by scottlocklin on Thursday April 09, 2020 @10:10AM (#22822677)
Sure: great -it can be used with other sorts of models as well, but it's really not that big a deal, and can be implemented in a lazy afternoon. I'm pretty sure lack of this as a first class language feature is holding back exactly no DL frameworks or programming languages, and glueing it onto Swift isn't going to make DL people use it. I've never used Swift's repl; I suppose that might actually be something that gets people on board.
[ Parent | on HN ]
Re:Swift: Google’s Bet on Differentiable Programming by stormtroper1721 on Thursday April 09, 2020 @11:01AM (#22823182)
> can be implemented in a lazy afternoon

Hi, I'm working on making a tensor lib in Rust (think numpy + autodiff) to learn about these topics. There isn't much information online about how projects like numpy and autograd work under the hood.

Do you have any ideas/tips/resources about how it could be done?

[ Parent | on HN ]
Re:Swift: Google’s Bet on Differentiable Programming by scottlocklin on Thursday April 09, 2020 @12:15PM (#22823976)
Numpy is basically lapack. You'd be hard pressed to replace that with something nearing its performance. For autodiff, I dunno, how about John Mount's explanation?

http://www.win-vector.com/blog/2010/06/automatic-differentia...

[ Parent | on HN ]
Re:Swift: Google’s Bet on Differentiable Programming by dietlbomb on Thursday April 09, 2020 @03:34PM (#22826316)
Do you know if Lush is still usable? I wonder what's stopping people from developing in it.
[ Parent | on HN ]
Re:Swift: Google’s Bet on Differentiable Programming by scottlocklin on Friday April 10, 2020 @08:53AM (#22832525)
It had a lot of cranky pieces when it made the 64 bit transition, and afaik it never got GPU support. I'm pretty sure it was ultimately abandoned because students didn't like it.
[ Parent | on HN ]