Re:Scallop – A Language for Neurosymbolic Programming
by alankarmisra on Saturday March 22, 2025 @04:33AM
(#43444367)
I'm wondering if this is a limitation though. If it can be learnt from training data, would it not be part of the neural network training data? I imagine we use Scallop to bridge the gap where we can't readily learn certain rules based on available data or perhaps we would prefer to enforce certain rules?
Re:Scallop – A Language for Neurosymbolic Programming
by daveguy on Saturday March 22, 2025 @06:27PM
(#43449158)
I'm pretty sure "differentiable" isn't necessary or sufficient to create valid and useful code.
Re:Scallop – A Language for Neurosymbolic Programming
by adastra22 on Saturday March 22, 2025 @10:33PM
(#43450484)
On the one hand, there are problems which by accident or design are nondifferentiable. Cryptography, for example.
In the other hand, these problems are routinely analyzed and solved by differentiable algorithms running on neural net substrates (e.g. you).
Re:Scallop – A Language for Neurosymbolic Programming
by daveguy on Saturday March 22, 2025 @10:48PM
(#43450540)
Nah, I'm not crackin crypto with my noggin buddy. Sorry.
Re:Scallop – A Language for Neurosymbolic Programming
by adastra22 on Sunday March 23, 2025 @02:43AM
(#43451251)
That's not what I was saying?
Re:Scallop – A Language for Neurosymbolic Programming
by sitkack on Sunday March 23, 2025 @05:07AM
(#43451704)
Papers are linked here https://www.scallop-lang.org/resources.html
https://www.cis.upenn.edu/~mhnaik/papers/neurips21.pdf
https://dl.acm.org/doi/10.1145/3591280
There is a 135 page book on Scallop https://www.cis.upenn.edu/~mhnaik/papers/fntpl24.pdf
I really love the concept. This isn't just differentiable neurosymbolic declarative probabilistic programming; Scallop has the flexibility of letting you use various (18 included) or custom provenance semirings to e.g. track "proofs" why a relational fact holds, not just assign it a probability. Sounds cool but I'm still trying to figure out the practicality.
Also worth pointing out that it seems that a lot of serious engineering work has been done on Scallop. It has an interpreter and a JIT compiler down to Rust compiled and dynamically loaded as a Python module.
Because a Scallop program (can be) differentiable it can be used anywhere in an end-to-end learning system, it doesn't have to take input data from a NN and produce your final outputs, as in all the examples they give (as far as I can see). For example you probably could create a hybrid transformer which runs some Scallop code in an internal layer, reading/writing to the residual stream. A simpler/more realistic example is to compute features fed into a NN e.g. an agent's policy function.
The limitation of Scallop is that the programs themselves are human-coded, not learnt, although they can implement interpreters/evaluators (e.g. the example of evaluating expressions).