Yann LeCun's paper isn't really a traditional ML paper, it discusses ideas at a high level but doesn't mention anything about implementation. If you were to try to create an RL agent based on his ideas, with the perception encoder and the perception-action loop etc, you would probably still use a large transformer to do it.