News for hackers. Stuff that matters.
Re:Deep symbolic regression for recurrent sequences by sytelus on Sunday January 30, 2022 @05:53PM (#30142102)
I don't think people here realize how amazing this is. You are providing very little data and the machine is able to fit the closed-form recurrent relation out of it. Very little data and the clean simple expressions is the key. It's just matter of adding more ops like sinusoidals, primes etc to make it recognize more and more complex expressions.
Re:Deep symbolic regression for recurrent sequences by verdverm on Sunday January 30, 2022 @08:14PM (#30143262)
People have been doing Symbolic Regression for a long time. I don't really see any novelty in this paper. They have left out a large part of the history and comparative results. How good is this compared to other methods?
Re:Deep symbolic regression for recurrent sequences by robrenaud on Sunday January 30, 2022 @10:31PM (#30144135)
Have people been using transformers for it? Their method beats mathematica by a large margin on OEIS.
Re:Deep symbolic regression for recurrent sequences by verdverm on Monday January 31, 2022 @12:31AM (#30144893)
There are much better options than mathematica for SR. The even cite the paper that made SR famous again (the nature one by Hod and Mike Schmidt).

They could have at least compared to them. They barely mention the standard method and leave out an entire class of top tier solutions, the more recent deterministic stratgeies. Shows they did not do their background reading and know the history. More like someone with a tool looking for an interesting application than someone deeply interested in a problem and finding all of the tools which can be used.

They go on to leave out all of the common test problems used in the field. Makes one wonder if the tests were too difficult for neural nets to produce publishable results with, or if they are just ignorant of the research and test problems. They almost state as much in the next paragraph. They don't even .mention how the other algos are capable where they are not.

It's also worth thinking about the environmental cost of the, now three, methods.

Re:Deep symbolic regression for recurrent sequences by amelius on Sunday January 30, 2022 @06:33PM (#30142413)
Wouldn't it become slower, the more functions you add? And exponentially slower if you start nesting those functions?
Re:Deep symbolic regression for recurrent sequences by CasperDern on Sunday January 30, 2022 @07:53PM (#30143068)
They used a fixed size transformer, where the vocab determines the functions and input/output range. So unless the model needs more 'memory' for your class of expression there wouldn't necessarily be a big change in performance. They have experiments in the paper with bigger/smaller vocabs.
Re:Deep symbolic regression for recurrent sequences by ShamelessC on Sunday January 30, 2022 @07:48PM (#30143029)
There's not any execution of the functions occurring here. It's modeled as a sequence-to-sequence problem where the training sequences have been pre-computed programmatically.

I think in this case the attention operation bounds the runtime by the longest possible sequence that can go in at once. Since you have a positional embedding, you can add arbitrarily many more functions without incurring a runtime penalty, although it may begin to interfere with the model's ability to effectively generalize as well, particularly when used with very long sequence lengths (so chains of operations, lists of ints).

Since each operation is mapped to an integer, the maximum number of tokens is bound by the precision of the system.