For people familiar with Gaussian Processes, it may help to think of Kalman filters as a special case of GPs where you can construct the inverse of the covariance matrix directly, and this inverse has a tridiagonal structure.

Thus, a really efficient Bayesian regression algorithm.