The implication is that the massive gains haven't been the result of any algorithmic breakthroughs, but rather have been due to the application of massive amounts of computational resources, which weren't available when current ML algorithms were invented. So far as I can tell, that's a true accusation. If you look at the papers coming out of Google and Facebook, they talk about throwing thousands of hours of specialized GPU (or even more specialized and expensive TPU) time at some of the problems. The advances have more to do with Moore's Law making brute force feasible than they have to do with algorithmic breakthroughs.

GPUs already existed when the idea to use them to make the feasible size of neural nets larger came about. For a long time the drive for the increase of GPU compute power was still gaming/commercial graphics houses. It’s really only in the last 1-2 years that we’ve seen highly specialised GPUs with features like tensor cores (or indeed google’s TPUs).

Also, calling neural nets ‘brute force’ because they use a lot of computing power to train a model is slightly reductionist - a true brute force approach to image recognition, ie enumerating all possible combinations of, say, 200x200x256x3 pixels, would be completely absurd and probably exceed the computing power currently available on earth.

It's not like neural nets are a new technology. They've been known since the '80s, at least. It's just that they were considered a dead end, because we didn't have the computational resources to run deep neural nets, nor did we have sufficient training data to make neural-net approaches feasible. Once those preconditions were met, neural nets took off in short order.

And you literally described it as a brute force approach in your comment.

Probably. Human intelligence is extremely fallible - based on the statistics the only reason we trust humans to do half the stuff they do is because there is literally no choice.

If we held humans to a high objective engineering standard We wouldn't:

* Let them drive

* Let them present their memories as evidence in a court case

* Entrust them with monitoring jobs

* Allow them to perform surgical operations

Humans are the best we have at those things, but from a "did we secure the best result with the information we had" perspective they are not very reliable. A testable and consistently performing AI with known failure modes might even be able to outperform a human with a higher failure rate (eg, we can reconfigure our road systems if there is just one scenario an AI driver can't handle).

Basically, you might be dead on the money that they are not 'trustworthy enough', but lets not lose sight of the fact that even being an order of magnitude from human performance might be enough after costs and engineering benefits get factored in. The weakest link is the stupidest human, and that is quite a low bar.

Ironically, the thing that is lost in this comment would be "accountability". In case of a human, you can go back / trace decision making criteria and hold someone accountable. In case of an algorithm, everyone washes their hands off. Performance is not the only criteria to make a decision if algorithms are "trustworthy" over humans.

There is so much more than one monolithic NN since those are easy to saturate in terms of precision and recall with enough training data and features, but are not enough to provide good UX in any complex domain/ontology. So it makes more sense to have many different models trained on each subdomain/taxonomy so that each can be specialized and then combined orthogonally.

Then the question becomes "how do we orchestrate them?" Well, there is a lot of research from the 80s and 90s that kinda got left by the wayside due to hype cycles (see the last "AI winter"). My faves are Collagen and Ravenclaw. And there is a lot of literature around topic frame stack modelling, which can be combined with various expert systems or other logics. I am currently using CLIPS (PyKnow) with custom Ravenclaw implementation. I believe b4.ai is doing something similar without the logic/rules engine, and actually applying ML to topic selection as well. My systems are goal oriented so I like to give them a teleology for business reasons, which would not suffice for AGI ambitions.

TLDR data science isn't enough on its own. We need engineers to architect things properly to solve problems.

Typically in a regression setting with an ensemble learner you're using a kind of weighted mean, where the weights are selected based on cross-validation performance. This is sometimes called a "super learner". See van der Laan, Polley, and Hubbard 2007.

Note that this suffers from being similarly awful in terms of theory as a lot of other ML stuff. It is not, for example, the case that two apparently similar datasets or problem domains will produce similar super-learner weights. Which is disturbing, because it's easy to believe that say SVM does better at X and penalized ordered logit at Y, but it's hard to believe that they both do better seemingly at random.

> Would you consider them trusthworthy in court, where lives are at stake?

We're using them at Generic Health Insurance Megacorp in production - lots of enterprises are. If you are in the IT industry, it might be useful to spend some lab time with ML. Possibly you have a misconception of ML and/or confuse it with AI.

https://news.ycombinator.com/newsguidelines.html

Also, please don't make single-purpose accounts on HN.

Indeed...

We have some strict(and irritating for profit-centric people) governance... one of the more interesting pieces of governance is called the "85/15 rule", which roughly translated, means that if we take in $100 dollars, the government mandates we use $85 of them to pay your costs, and $15 to pay our staff, light bills, and any other expense we have. If we end up only using $80 to pay your costs, we have to refund the remaining $5 to your group plan.

Here's the obvious secret about health insurance that people like to have conspiracy theories about..I can't speak for other institutions in other countries, however...our stance is really simplistic: you can't pay premiums if you are not alive, therefore it is in our mutual interest for you to remain alive. All the conspiracy theories such as "but you don't want that cancer patient in your insurance group plan!" are just that..conspiracy theories. We absolutely do want that person in the group, because then that group's rates go up! The costs for that patient's care are more or less fixed(and known), built on the assumption of a terminal outcome. We're going to pay for it anyway, and try to make that miserable experience as pleasant as possible for everyone involved. That type of service is how you get repeat business, and a good reputation.

This likely falls on deaf ears. Feel free to return to the zealous insurance hatred, and I'm going to return to writing code. Not for death panel machines. Promise.

I'll only make a small comment about Catholic priests, since I was at one time a Catholic. The problem wasn't just some of the priesthood was into pedophilia, but that when the church was made aware, actively covered it up.

> one of the more interesting pieces of governance is called the "85/15 rule"

If I remember correctly, that was passed via the PPACA. And it is also in jeopardy with the continual "repeal and replace (with nothing)" procedures since the PPACA's passage and SCOTUS failed challenge to dismiss. I believe there is a current federal court case with 20 states or so suing on grounds of constitutionality. And with the makeup of SCOTUS now, has a good chance of having the whole law deemed unconstitutional.

> Here's the obvious secret about health insurance that people like to have conspiracy theories about..I can't speak for other institutions in other countries, however...our stance is really simplistic: you can't pay premiums if you are not alive, therefore it is in our mutual interest for you to remain alive. All the conspiracy theories such as "but you don't want that cancer patient in your insurance group plan!" are just that..conspiracy theories. We absolutely do want that person in the group, because then that group's rates go up! The costs for that patient's care are more or less fixed(and known), built on the assumption of a terminal outcome. We're going to pay for it anyway, and try to make that miserable experience as pleasant as possible for everyone involved. That type of service is how you get repeat business, and a good reputation.

My anger, as well as many other peoples' anger, is the fact that this system is opaque. I go to a doctor, and have procedure/drug prescribed, and there's this song and dance about "preapproval", "permission" and all other sorts of roadblocks. Whether the medical insurance company is for/non profit doesn't matter too much to me. All I know is that the medical insurance is sitting between me and my doctor and making decisions about my care without a medical degree and no patient-doctor association.

And the moment medical insurance is taken out, the prices go up by 10 fold. That's mot the medical insurance companies' fault... But that's the end result for us. And medical insurance companies become de-facto arbiters of patients' health. Again, when questions are shoved in this black box, magic answers come out.

And what I was criticizing is that the use of AI in this context means that the decisions are now truly black-boxed, rather than just a process of actuarilists (sp?). That was subpoena-able and discoverable. The fact that some neural network algo was trained on GBs of data and outputs magic weights of "accept or deny" is an anathema. Those decisions should be understandable. Those decisions should be defensible (as long as we have a profit-based medical system).

Even decision trees would show traceability of how an input got the appropriate result. And if there were questionable or illegal things in there, then they could be challenged or changed.

> This likely falls on deaf ears. Feel free to return to the zealous insurance hatred, and I'm going to return to writing code. Not for death panel machines. Promise.

Not at all. I do have grievances with how the US does medical, and insurance is only one part of the whole. I come from a point that we should have health provided by tax dollars. We as a nation already spend triple what France does peer capita, yet only a small fraction gets care. Simply put, too many people slip through the cracks. I wouldn't say it's a zealous hatred. It's a well informed long-stewing anger that people who are ill can't get help/fixed.

Thank you for the discussion :)

That's how it always has been and always will be until perhaps one day there is truly nothing 'truly human' left to slice off.

https://news.ycombinator.com/newsguidelines.html

Not to mention that part of what makes NN such a step forward is precisely the high nonlinearity. When you have millions of parameters, the contribution of each is unimportant and so any kind of analog to ANOVA would be barking up the wrong tree. In a NN all those parameters work in concert to learn a decision boundary to separate data. It’s not intelligible at the minute level but at least we know what it’s doing in the end. The problem of course lies with the outliers and that’s not so much a problem with NN being a black box as it is a problem with the nature of large datasets and our own inability to rationalize each and every datapoint.

I’m going to defend NN as the natural evolution of regression. It’s precisely their high nonlinearity which makes them better. The problem is not that they “are” alchemy but that we treat them “as” magic. Society as no place leaving important decisions to algorithms unintended, NN or not. If major insurance companies left their decisions to logistic regression (which was and is still the case), then would we be making the same arguments? Probably not because someone paid by the insurance company will pull out that other kind of alchemy called ANOVA ...

Yes, somebody can produce "magical" regression models with terms divorced from reality. But unsupervised learning is never guaranteed to produce a reality-grounded model, no matter the user's skill. It goes through steps, tries different transformations, and chooses the algorithm which led to the best result. Logic and understanding played no part. That sounds like alchemy. It definitely works, but alchemy also stumbled onto theories later explained by chemistry.

And, yes, anyone who doesn't consider the implications of the linear model behind ANOVA would also be a "magician."