I wonder how well it would work if llms would do "online learning". After every interaction they'd be asked to summarise new knowledge (assuming the input is trusted) and create multiple variations on it, then train on it. I wonder if such a model would improve or degrade over time.