So the logical follow up here is, since the model can obviously interpret the weights in ways we cannot, to simply take the weights of some LLM itself and then write a program and algorithm that just implements the algorithm the LLM follows. It would seem that, since the language models are obviously over-parameterized, such a system would be able to be written in a 'conventional' way with much simpler code.