In the past couple of years, several papers have shown that predefined symbolic relationship can improve over vanilla DL. For example, recognizing a picture of numerical arithmetic equations and compute the result. This is very difficult for neural networks to parametrize over the pixel space.
Moreover, statistical generalizability is currently all derived from concentration theories. This means that the knowledge encoded by a neural network model depends on the statistical distribution of the data. If two people have two different sets of data, they might end with two very different models. Symbolic generalizability is quite different. A rigorous mathematical proof holds true as long as we all live in the same world with the same set of axioms. For example, no person can appear in two physical places at the same time. Or, if A causes B, A has to occur before B occurs. These knowledge can't be learned through statistical methods with no symbolic priors. We postulate the logic first, then verified through observations.
At last, problems that statistical learning handles well so far are essentially interpolations of the collected data. Being able to extrapolate well is still an unknown. Will inductive logic programing work better in this scenario?
Symbolic is the signature of human intelligence. All of our scientific breakthroughs are encoded by symbols, even the DL (deep learning) stuffs. It won't be replaced by the numerical paradigm completely in any time soon.
As to why that is, my best guess is a combination of not having enough man-hours (we're still a relatively small company) and how difficult it has historically been for people to jump in and play with Cyc. There could also be a cultural lack of awareness that people still have interest in tinkering with it, which is something I've thought about bringing up for discussion.
As to the accessibility issue, that's been one of our greatest hurdles in general, and it's something we're actively working on reducing. The inference engine itself is something really special, but in the past most of our contracts have been pretty bespoke; we essentially hand-built custom applications with Cyc at their core. This isn't because Cyc wasn't generic enough, it's because Cyc was hard enough to use that only we could do it. We're currently working to bridge that gap. I'm personally part of an effort to modernize our UIs/development tools, and to add things like JSON APIs, for example. Others are working on much-needed documentation, and on sanding off the rough edges to make the whole thing more of a "product". We also have an early version of containerized builds. Currently these quality-of-life improvements are aimed at improving our internal development process, but many of them could translate easily to opening things up more generally in the future. I hope we do so.
That meshes with what I've heard at conferences, that Cyc management was worried people were treating OpenCyc as an evaluation version of Cyc, even though it was significantly less capable, and using its capabilities to decide whether to license Cyc or not. The new approach seems to be that you can get a free version of Cyc (the full version) for evaluation or research purposes, and the open-source version was discontinued.
https://v1.probmods.org/learning-as-conditional-inference.ht...
"Cyc gains an intuition about what kinds of paths of reasoning to follow when searching for conclusions"
The possible paths come purely from symbolics. But that creates a massive tree of possibilities to explore, so ML is used simply to prioritize among those subtrees.
> I (and my company) believe in a hybrid approach
> I don't know of any higher-level hybridization experiments
That contradiction and the admission that Cyc only has "one ML person on staff" signals to me, an outsider, that the belief of parity between Machine Learning and "Symbolic" might be predicated more on faith than on reason.
https://news.mit.edu/2019/teaching-machines-to-reason-about-...
We aren't an ML shop ourselves; we don't claim to be. Given that we have around 100 people, we focus on what we have that's special instead of trying to compete in an overcrowded market. The idea of hybrid AI is something we see as a future part for us to play in the bigger picture of machine intelligence.
But I will affirm that Cyc is fundamentally symbolics-based. We don't position ourselves as anti-ML, because it's seriously good at a certain subset of things, but Cyc would still be fully-functional without any ML in the picture at all.
I'd love to experiment with an automated planner (a good old symbolic AI technique) but use deep learning to design the heuristic. It feels like a lot of reinforcement learning techniques are becoming close to this kind of things.
AlphaGo was a rough implementation of that. Do you know of some efforts from symolic AI in that respect?
And do you still accept candidates ? :-)
> And do you still accept candidates ? :-)
If you mean job candidates, then yes, we definitely do!
I am employed now but I'll probably send a CV when I am looking for a change!
I (and my company) believe in a hybrid approach; it will never be a good idea to use symbolic AI for getting structured data from speech audio or raw images, for example. But once you have those sentences, or those lists of objects, symbolic AI can do a better job of reasoning about them. Pairing ML and symbolics, they can cover each other's weaknesses.