The art of programming and why I won't use LLM

Re:The art of programming and why I won't use LLM by trueismywork on Sunday August 25, 2024 @08:14PM (#41352707)

Thanks for your insightful comment. I'll read the links later.

I had just assumed that RNNs were TC, didn't think of limitation put on by bounded precision since I assumed that any bounded precision could be compensated by growing memory module.

So, after your comment, I went literature searching and I found this: https://papers.nips.cc/paper_files/paper/2021/hash/ef452c63f...

I haven't read it yet. But if it is true, then RNN would seem to be TC

[ Parent | on HN ]

Re:The art of programming and why I won't use LLM by nyrikki on Sunday August 25, 2024 @08:41PM (#41352851)

Note from the open review of that paper.

> As discussed in the paper and pointed out by the reviewer, the growing memory module is non-differentiable, and so it cannot be trained directly by SGD. We acknowledge this observation.

Two stack FSA/RNN are interesting, but as of now, not usable in practice.

[ Parent | on HN ]