You are describing a recursive Bayesian approach, which can have significant computational and storage advantages for filtering (for example, Kalman filters). For this to work well, the prior must be able to adequately represent the learning of the posterior, which may be practical with a self-conjugate prior or a Monte Carlo approximation such as what particle filters use. In practice, for nontrivial machine learning applications, self-conjugate distributions rarely model the problem well and good approximations of the posterior into a concise prior are rarely practical.