There is an obvious overlap between programming languages and machine learning at Google: TensorFlow.

For example, see "A Computational Model for TensorFlow" https://research.google.com/pubs/pub46196.html

There are tons of interesting things going on in TensorFlow from a programming language perspective. It has optimization at the low level, like CUDA and SIMD back ends via the Eigen library [1] (which is pretty crazy C++ metaprogramming in its own right).

But it also has optimization at the high level / cluster level, e.g. deciding which nodes to put computations on, to minimize data movement across various networks, etc.

It also has multiple front ends. Python is the main one, but IIRC people were developing others (maybe not at Google).

I worked adjacent to the TensorFlow team, and A LOT of people had BOTH ML and PL skills [2]. It's not an either-or thing. It's best when you have the same person with both sets of expertise.

I think a big problem with large parts of the academic PL community is they're not exposed enough to real applications. Difficulty isn't proportional to real-world benefit. It's certainly difficult to invent type systems to statically detect minor problems, but that doesn't mean it's important for creating and maintaining software. (Sorry, had to rant about that.)

Machine learning is a domain rife with programming language problems, but of course it takes a long time to develop that expertise. I'm sure Lattner would be a good person to synthesize knowledge in the different domains.

[1] http://eigen.tuxfamily.org/index.php?title=Main_Page

[2] edit: I should really say ML and distributed computing skills. But most people with distributed computing skills know a decent amount about programming languages; they overlap in MapReduce-type big data frameworks too.

Jeff Dean got his PhD under Craig Chambers doing PL work; Sanjay Ghemawat was also a compiler guy, they went on to mapreduce and I think, at least Jeff Dean, is heavily involved in Tensorflow.

Google hires a lot of PL PhDs to do work that isn't very related to PL (we joke about PL PhDs working on ads a lot). One could guess that they just make good developers because of their experience, but the specific knowledge might not be much in demand.

Right and Craig Chambers later joined Google. He led/developed Flume [1] (a Java framework on top of MapReduce) and the unfortunately named Cloud Dataflow [2] (which aims at unifying streaming and batch computation)

I say unfortunate because pretty much all the big data frameworks could be called "cloud dataflow", including TensorFlow itself.

People using Google Cloud might recognize the name of VP Urs Hoezle, who worked on the OO language Self in the 90's with a lot of the same people [3].

And AFAIK that's where v8 came from... Urs hired his former colleague Lars Bak and told him to write a fast JavaScript interpreter for Chrome.

[1] https://research.google.com/pubs/pub35650.html [2] https://research.google.com/pubs/pub43864.html [3] http://dl.acm.org/citation.cfm?id=619798

Wasn't V8 developed by Bak & co in a separate company, which was bought by Google? (Didn't find any source, but based on what I thought I read in the news at that time ...)

v8 wasn't developed by a separate company, but Bak did bring at least one of his colleagues along to Google (I'm pretty sure just as new employees to Google). And he had another company that worked on a SmallTalk VM called OOVM, targeted at embedded devices, right before he joined Google:

https://astares.blogspot.com/2012/04/inside-oovm.html

However it apparently wasn't acquired by Google:

http://www.businesswire.com/news/home/20040727005088/en/Esme...

Other discussion:

http://forum.world.st/OOVM-details-td4561492.html

> There is an obvious overlap between programming languages and machine learning at Google

This doesn't explain his very brief stint at Tesla as head of AutoPilot.

Read his resume, that will explain it very well.

The question was why a PL developer has seemingly turned away from the domain in which he's prominent, to work on machine learning, where he has no prominence to my knowledge. His resume does not explain it very well, or at all.

> which is pretty crazy C++ metaprogramming in its own right

Expression templates aren't that crazy, in the grand scheme of C++ meta programming.

But they are the ur-crazy.

Not even close. At the very least, you have to consider APL.

Makes me wonder if Google is working on a programming language for machine learning...

That's what TensorFlow is. It happens to use the Python interpreter as the front end, and for metaprogramming, but the language has its own semantics (see the paper above).

You could probably an invent your own syntax for it (and I'm sure someone has), but that's a small part of the picture.

Interesting idea, indeed. Add to this programming language:

1.True multithreading.

2.Native support of spread computation to heterogeneous computation devices, across CPU/GPU/Whatever-Chip-You-Call

3.Auto vectorization.

4.PyTorch style autodiff, and should be able to JIT hotspot as the program runs.

5.Native distributed support with built-in primitives.

6.Visualization as standard library, even language primitives.

This actually blows my mind. I know that probabilistic languages exist, but something very interesting could be done here.

It may have to do with the fact that he just wanted some new challenges to sink his teeth into. As an old school hacker myself, I can relate. Even really cool technologies start to feel mundane and tedious after you've done them for long enough.

The real challenge with senior positions is adjusting to new environment/dynamics, not to mention the expectations that come from "legend" status. I'm not a legend, but I certainly had a much easier time in the earlier stages in career just focusing on the tech direct. When you're the hot new hire, a certain group at the new company expects a lot form you, while others who have been there longer think it's just hype. (I'm assuming something similar happened at Tesla).

Wish him all the luck on the new adventure! And being an AI geek myself: NO AI cannot solve everything. Not even close. (at least for now)

Sentient machine conversations are straight up funny to me. Still, I can't help get over the cool/geek factor of working on Machine Learning/Deep learning or exploring new algorithms that could significantly improve the field.

If you want to understand his motivations, he gave a wide ranging interview to the ATP (Accidental Tech Podcast) guys at the time he jumped ship to Tesla.

Basically it seemed to boil down to him now being a people manager as much as he is an engineer - he really enjoys helping teams of people succeed at complex technical problems. This doesn't surprise me at all, given how the nature of work often changes as one climbs an org tree.

http://atp.fm/205-chris-lattner-interview-transcript/

As a machine learning researcher I have to say that you should join us too! AI is the future!

PL is an area with big challenges, but AI has even bigger challenges and the impact in the society is bigger, much bigger, too. I would change the question and ask why anyone shouldn't be interested in AI?

Because not every problem can be solved by AI.

I'm more interested in IA (intelligence augmentation) actually, which seems to overlap with PL nicely.

Not now, but if you can really create AI, it shouldn't be indistinguishable of human Intelligence.

My point is that it is unlikely to make big steps now to advance in PL, but it is very likely you can do it in AI. The last 5 years have been crazy and the people working on it now is much more than few years ago.

Note: most probably you wont be getting human-like AI anytime soon

> Not now, but if you can really create AI, it shouldn't be indistinguishable of human Intelligence.

If we really create AI that is "indistinguishable of human Intelligence", then all questions about human jobs or research become moot, no?

If someone was to create AI that is human-equivalent (aka strong AI) then that would be their final project as a human being. There would be no need for human professional or academic effort after that point.

>'If someone was to create AI that is human-equivalent (aka strong AI)'

Creating AI with human level intelligence isn't going to be so great, most people are complete idiots. Creating AI with above human intelligence is a different story.

A well-known "paradox" states that tasks that require savant-level intelligence from humans (like playing chess) are actually doable with computers, but tasks that any idiot can do (say, pouring yourself a cup of tea) are crazy hard.

Well... They said the same in the industrial era. There are a lot of things to consider, for example:

- AI might not like to work, as we don't like to work

- You suppose that if you create AI then the AI can create a more intelligence AI and this to the infinity. We don't know if the intelligence is unlimited or limited. We might have reach the top intelligence. (note: the more intelligent the brain the slower it is, and a bigger brain does not relate to intelligence)

Sounds a lot like saying people should get into management so that they can get other people to do stuff for them. People can do anything people can do, so you should get into people.

And it's totally true! A good manager can make anything happen that their team can accomplish, and their impact can be huge. But some people want to actually work on the stuff, not work on the stuff that works on the stuff.

Actually, compilers are a great example of this dynamic - when you work becomes more and more impactful, but more and more removed from actual problems people have. Say, you write a script to solve a specific problem someone has. But that is not very impactful - one person is happy and that's all. So you generalize your program to a whole class of problems. But that's not the end. Next, you write a framework for writing programs similar to the one you've developed on the previous step. And of course, all languages being shit, you'll want to write a compiler for a language to write your framework in. So you end up with a thing that is very impactful but also pretty useless on its own.

Nice example. Really, lots of technology is like this. If you enjoy arithmetic, you do it. If not, you use a calculator. If you enjoy digging, grab a shovel, otherwise use a backhoe. Nothing wrong with picking the layer you enjoy, and a backhoe isn't always the best answer anyway.

PL seems easier to get into and make useful things with, like a static analyzer that enforces some rule at a big co. From my perspective it seems like your need something PhD level to do useful things with it and get paid better than a standard FANG software engineer.

I'm probably wrong although, it's probably an exposure thing.

Well as another machine learning researcher, I think you can make a major impact working on problems that aren't in AI.

Now if you are a PL person interested in AI we would love to have you and your expertise!

Oh and if you want to use AI to solve PL problems like program synthesis and program completion, that's cool too.

Haha I was going to say, solving synthesis via PL+AI is way more exciting than either technique alone.

-- A former PL researcher making AI way more useful via vanilla PL techniques

Welcome to the party!

He is still on Swift core team so why you think is he is not interested in Compilers/PL? A person of his/her caliber should be able to work on multiple things with very high level of performance.

Google has its own GPGPU compiler: https://research.google.com/pubs/pub45226.html

It's actually all open-source in clang: https://www.youtube.com/watch?v=KHa-OSrZPGo. Try it out!

That's a great talk. Thanks a lot!

Well he already said in a couple of interviews something like "I write C++ so that you don't have to".

Also a few other Swift team members used to be part of ANSI C++ comitee.

Don't you get bored doing the same thing for more than a decade?

I'd expect Google is looking to extend XLA: https://www.tensorflow.org/performance/xla/

Well maybe he realized Swift is a terrible programming language he should do something else.

(Yes I know, he's still involved in Swift. Its a joke. Though Swift really is a terrible programming language).

AI/ML is the most exciting fields in CS now. It brings challenges on so many levels: algorithms/software implementation/hardware(client/server/network communication)/resource placement(heterogeneous computation ), and the list goes on. I would go as far as to say it has the potential to ascend as the new purpose of computation itself. Hardly blame someone jumps the ship to goldmine this new wild west, however established that person in his own field may have become.

I'm finding that some traditional developers nowadays are more and more interested in AI/ML, rather than just plain coding. Maybe they're able to tell which way the wind is blowing a little better than most of us? After all, one of the applications of ML will be to learn how to write software. It could very well put traditional developers out of a job.

PL / compilers has a lot to do with logic and formal reasoning, so I wouldn't be surprised if he's interested in applying that to other domains. Merging DL with more traditional reasoning frameworks is an interesting question I know a few people have tried attacking.

Of course, that's just speculation.

After having been working on compiler and PLs for more than a decade, you would feel bored and want new challenges. Besides, Chris did not leave PL, he is still involved in Swift, LLVM etc.

Maybe he'll create a higher AI language for TF. That'd be really really cool.

Quite possible that his last contract's non-compete still makes that hard...