News for hackers. Stuff that matters.

A comment on Refact LLM: New 1.6B code model reaches 32% HumanEval and is SOTA for the size. Parent comment

Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by KRAKRISMOTT on Monday September 04, 2023 @02:00PM (#37383131)
> We’re using formal logic in the form of abstract rewrite systems over a causal graph to perform geometric deep learning. In theory it should be able to learn the same topological structure of data that neural networks do, but using entirely discrete operations and without the random walk inherent to stochastic gradient descent.

Abstract rewrite like a computer algebra system's (e.g. Wolfram) term rewriting equation simplication method?

[ Parent | on HN ]
Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by taywrobel on Monday September 04, 2023 @02:48PM (#37383599)
Heavily influenced by Wolfram's work on metamathematics and the physics project, in so far as using a rewrite system to uncover an emergent topology; we're just using it to uncover the topology of certain data (assuming that the manifold hypothesis is correct), rather than the topology of fundamental physics as he did.
[ Parent | on HN ]