The big lift with LLMs, causing them to really explode in real-world capability, has been tool calling. Dumping data into the LLM only really works if the data fits very, very comfortably in the context window. The way the best harnesses work is that they provide the LLM a bundle of tools to manage context and memories on its own, and let the LLM use these tools to manage its understanding of the world.

The problem with these Garmin and Strava AI summaries is that most of them just point an LLM at a heap of data and ask them to offer a summary. They also usually use older models since the feature is not something particularly well curated. I do financial modeling with LLMs and use the LLMs to build/verify/iterate on models.

If you're curious on what it takes to run long-running tasks Anthropic has https://www.anthropic.com/research/harness-design-long-runni... out.