Re:Can a Transformer Represent a Kalman Filter?
by gautamcgoel on Wednesday December 13, 2023 @08:26PM
(#38636664)
Yes. Also the universal approximation theorem won't directly yield the kind of "strong" approximation we prove, which is uniform in time and holds for an arbitrary (possibly adversarial) sequence of disturbances.
Speaking of questions, it seems to me that in light of the universal approximation theorem, the question isn't "can a transformer represent a Kalman filter," but can it do so simply and somewhat efficiently. That seems to be what you're getting at here, yes?