Commercial Mixed Integer Programming solvers can optimize massive problems to provably global optimality very very quickly, and they too have an internal learning mechanism (cutting-planes).
They excel at reasoning but they can only learn from the outcomes of the rules they are provided and how they interact, they cannot learn from data. With the advent of soft-constraints (which have an associated weight) I wonder if such systems could be adapted to learn probabilistic rules as well.
Some existing systems (like AlphaGo) already combine symbolic approaches (Monte-Carlo Tree Search) with Deep Learning, but what I'm thinking about here goes beyond that.
Anyway I'm not an expert on SAT solving. Thank you for the perspective you provide with which I was unfamiliar.
Edit: I don't think the OP meant SAT solvers when they said "symbolic learning"?
SAT solvers, however, learn from their "mistakes", at least internally, which is really interesting and I think opens some very, very interesting research questions that as far as I know don't have that much money going into.
For example, human brains can reason internally and learn in a similar manner, learning from "mistakes" of rule application in the same way (i.e. if I try to do this, then that fails, but if I do this other thing, then it works but not fully...). Just some food for thought.
Automated Theorem Proving is also a very important success of the Symbolic camp but I would argue a less interesting one, since SAT solvers can do much more than prove theorems (and ATPs that can do more than prove theorems invariably use a SAT/SMT solver internally, like Vampire does for example). SAT solvers (and extensions) are used in model checkers, software and hardware verification tools, software and hardware synthesis tools, operations research, etc.
One reason I think that theorem provers are more important than SAT solvers is that SLD-Resolution at least can be efficiently implemented.
Also, Resolution can be used for induction, from examples and a background theory. That's a relatively recent result and you won't find a very clear account of it in the literature, but for instance, see here some early work:
Meta-Interpretive Learning of Higher-Order Dyadic Datalog: Predicate Invention Revisited
https://www.ijcai.org/Proceedings/13/Papers/231.pdf
You have to unpick it from the language used but "meta-interpretive learning" means that it's based on a Prolog meta-interpreter, so an implementation of SLD(NF)-Resolution. And those "metarules" are really second-order definite clauses. The approach is really higher-order SLD-Resolution that turns out to be inductive, rather than deductive.
But what you say about learning from "mistakes" reminded me of a rival approach, "Learning from Failures":
Learning programs by learning from failures
http://andrewcropper.com/pubs/popper.pdf
Which is, incidentally, implemented by means of SAT solvers (via ASP, ultimately). I'm more of a fan of the meta-interpretive approach (I study it), but I think the Popper paper might interest you, judging from your comment.
Vampire uses a portfolio of strategies, including the application of an SMT solver (Z3). This link goes into some detail: http://smt2019.galois.com/papers/tool_paper_20.pdf
This is the most relevant bit for our conversation:
> There are only two cases where Vampire can return sat: Firstly in UF and secondly, if Vampire produces a ground problem after preprocessing it may pass this problem to Z3 and report its result (possibly sat) directly.
Essentially this refers to model building, i.e. solving existential problems. That's where SAT solvers excel. This includes producing counter-examples which are extremely useful in industry.
I totally agree that there is room and even a need for more symbolic systems within deep learning but I'd argue that you can't at this point do away with the "deep layered" approaches.
The examples you cited are very important achievements, especially the very early ones, but I think they also show that they are also very limited in a lot of ways. For example, expert systems found a niche, but they still had a very hard time with edge cases and learning which imo is essential to intelligence. More traditional logic based algos can vastly outperform say, neural networks in a lot of situations but only when the problem space is in a way "known". Plus, the GOFAI school used to promise a lot, lot more than what those performant but usually hyper specialized systems ended up doing.
I see that my comment could come off as disrespectful for what was accomplished before. But it really isn't!
It's just that I don't agree with the "nostalgics" who usually dismiss the modern approaches and idealized some sort of symbolic vision of intelligence. Those aren't common, and most of my "old school" professors were just as excited by deep learning. But there is a vocal minority imo who are viewing the past with rose tinted glass, when I don't think it's controversial that there is no real way for traditional ("pure") symbolic AI to end up achieving either general intelligence or to outperform deeplearning with finely tuned hand crafted logic.
Yes, "GOFAI" overpromised and underdelivered and that was a major reason for the two AI winters that essentially destroyed the field by freezing funding and shrinking research positions and output.
Personally, I'm neither nostalgic of older approaches, nor dismissive of modern approaches. The important thing is to have a clear understanding of the capabilities available, regardless of approach. It's obvious to me that older systems could do things that modern systems can't do (principally, reasoning and knowledge representation) just as modern systems can do things that older systems couldn't do (learning). However, there are approaches that bridge the gap, such as symbolic machine learning], like the approaches I study that learn logic programs from examples using theorem-proving techniques. There is also, of course, continued research in other branches of symbolic AI, like planning and SAT solvers, that seem to have made great progress in the last years. I think the worst that can happen now is to nip such research in the bud by denying it funding just because it's not deep learning.
Gary Marcus' article quotes Emily Bender about how overpromising, this time by the deep learning community, "sucks the oxygen out of the room" for other kinds of research. This is apposite. Research can't become a monoculture, otherwise the ability to innovate will disappear. For innovation, there must be diversity of ideas. The risk I see right now is that such diversity will be lost and that, in the long run, progress in machine learning will stall. Throwing out everything that was learned in the fist 50 years of AI will not help anyone avoid the mistakes of the past, for sure.
The first system to win against a Grand Master in chess was Deep Blue, which, despite its name, was not a deep learning system, but a symbolic, hand-crafted, rule-based, system with alpha-beta minimax, using a hand-crafted evaluation function and an opening book of moves [1].
The first system to dominate human players in draughts (checkers) was Chinook, also a hand-crafted symbolic rule-based system [2].
The first system to outperform experts in medical diagnosis, in particular, diagnosis of infections, was MYCIN, an expert system [3].
The first system to win against a human in the question game Jeopardy was the much-maligned, but symbolic-based, and very successful in that one task, Watson [4].
SHRDLU was an NLP system used to direct a (virtual) robot hand, whose capabilities remain unsurpassed by modern systems [5].
The performance of symbolic systems remains unsurpassed in various tasks such as classical search, classical planning, SAT solving, automated theorem proving, program synthesis, etc.
I assume that by "symbolic learning" you meant symbolic AI in general, btw. If you're talking about symbolic machine learning in particular, it is worth noting that decision tree learners, like CART, ID3 and C4.5, which have been wildly successful in machine learning, data mining and data science for many years, are symbolic systems that learn a propositional logic "model" (a theory).
__________
[1] https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)
[2] https://en.wikipedia.org/wiki/Chinook_(computer_program)
[3] https://en.wikipedia.org/wiki/Mycin
[4] https://en.wikipedia.org/wiki/Watson_(computer)#Comparison_w...
[5] https://en.wikipedia.org/wiki/SHRDLU