As much I'm hoping there'll be a breakthrough in AGI, maybe the right approach is the one AlphaGo was using: DL not as the top level decision-making, but plugged into a traditional decision-making algorithm in specific places.

Yes. Though arguably this is just the same thing as the traditional approach to deep learning, which is to select features to input into a model. You don't always just have to train on raw information, pre-processing the input data and calculating specific features that we know, as humans, will be important for the final result and feeding those in addition to the raw data is a common approach. Don't see much difference between this and taking the output of a network and running it through a few decision trees. Most publishable projects applying AI you see typically have these type of human interaction on both sides of models.

> As much I'm hoping there'll be a breakthrough in AGI

I think it probably won't be one breakthrough, but several, over decades. Personally, I'm pretty happy that AGI is taking a long time to materialize. We likely won't see a "fast takeoff scenario" (the computer is learning at a geometric rate !!1). It will likely happen gradually over years (progressively more intelligent, more aware computer systems), and we may have a chance to adapt in response.

A business professor told me that cars were entirely incremental innovation all the way from the model T. Just little improvements, one at a time. I don't know if that's true, but I wonder if it will be an apt analogy for AGI - one feature at a time, and older attempts at it just look outdated.

Perhaps, although the automobile has always had a clear explicit purpose: convert potential energy into rotational energy in a controlled manner. It’s hard to identify such a singular purpose for AGI.

Things like automatic transmissions and fuel injection aren't particularly incremental.

As someone who once got stuck in an intersection after flooding a '79 Monte Carlo's carburetor while deciding to go on red, I'm with you on fuel injection. But I could see the counter-argument that something that makes an experience nicer is not radical innovation. That old junker got me where I need to go for a while.

It’s the classic: ML + business logic

I think that’s fair, deep learning today has an issue with learning guide rails and obviously it is only as good as the data you feed it. I think it’s fair that our models need more

Making 90% of your income off of this tech over the last n years is different than that tech being successful. I work at a very large company that is trying to use ML and AI in all kinds of places. The trend I am seeing is that most of that effort is falling flat, really flat, in fact. They have success in places where regular algorithms would also succeed, but just having regular developers design matching systems and do pretty basic statistics isn't sexy in terms of marketing, so they hush it all up and pretend that ML and neural nets and things are the only way forwards. I don't think it is. Our problem is that we have too many ETL robots who aren't very intelligent people and not very forward-thinking themselves!

The cutting edge NLP stuff just showcased at my company was pretty lame, too. I barely saw any statistically significant results at all and yet they rather unscientifically proclaim success because they got any effect at all. Some of what we do in our field doesn't matter because it comes down to whether a customer got a 2nd call back and got converted to some minor sale or added to a program for them. It's throw away and creates good will at conferences and talks. We make a big deal out of it.

We are spending hundreds of millions on projects, trying to save money on generating leads, reducing interactions with customers and vendors through staffed phone banks, and so on. My company has hired all kinds of academics and research type people and has given them titles of "Distinguished this" and "Principal that" and honestly there's not that much to show for it, maybe zero direct outcomes so far. What galls me the most is in all the conferences and demos they are showing off things like High School robotics vehicles and AI parlor tricks and astonishingly little has translated into the business we do. Meanwhile, there are people in the company who do know how to reduce costs and get more done and have outstanding outcomes, but their techniques are not sexy and thus unimportant to the PT Barnum MBAs running our company. I'm sure that's true most everywhere, of course.

These Principals and Distinguisheds all keep proclaiming success while cashing fat paychecks. Meanwhile this year, our stock has had a tough go of it, so I'm curious whether these attempts will continue. The market takes no prisoners. Sure we get a lot of mileage out of looking cool for the recent grad crowd purposes of recruiting--kids want sexy, cool tech projects to work on and words like "insurance" turn them off, so there's that, I guess.

My take on that is that it won't be long before all those new recruits will figure out they got bait and switched pretty bad and that they aren't going to get to work on any of this sexy ML and AI stuff anymore than I am in my role. I got lured in by Data Science (because PhD), which just shows how gullible I am, but at least some of that traditional statistical modeling is having an impact here and there. The problem again is that even that is overblown by a couple of orders of magnitude! In my project, we're simply trying to get more real-time data out to people who need it without having to call in to get it and that is ridiculously difficult because of all the systems we try to knit together and how overall terrible our data quality is. And now my boss wants to build out an "analytics engine" to capture some of this sexy ML and AI stuff. It leads me to believe that the people involved are most interested in getting promoted and not much more.

Anyways, it is cool tech, but American taxpayers and people who are forced to buy our products are paying for it and I rather think they would prefer to spend their money in some better fashion.

Thanks for your response. Sure, the level of hype is rather high for DL.

That said, I also lived through and worked through the level of hype around expert systems. I think the high level of hype around expert systems in the 1980s was much more extreme and unwarranted that the DL hype levels. I base this on selling expert system tools for both Xerox Lisp Machines and for the Macintosh when it was released in 1984. Some of my customers did cool and useful things, but nothing earth shaking.

At least DL provides very strong engineering results for some types of problems.