Recall has lost a ton of useful metadata you already have - both URL visits and streaming are clearly discernible actions, both at the network stack level, and from your browser history. Throwing that away to trust an LLM to re-infer the same data is both reducing data fidelity and significantly increasing processing cost.
If you want to see this done reasonably well, I'd suggest looking at e.g https://beepb00p.xyz/promnesia.html (which not surprisingly bears a strong similarity to what the article discusses)
LLMs don't add much value here, outside of tightly locked down systems where screenshots are the only way of exporting.
The value the LLM adds is interpreting/processing data without having to tailor input streams. Imagine if formats change, fields get renamed, and so on. The maintenance would be a headache if this was done on a per-service level. I think the reduction in fidelity seems like a reasonable tradeoff, but that's for the user to decide of course along with local/cloud processing and proprietary/open source software.
Even things like invoices from the same service change format over time.
A prompt every few minutes that would ask "What are you doing now?" would be interesting to me, as a professional procrastinator. Maybe an even better one would be one that says something like "In the last 10 minutes, you spent 90% of it on Hacker News".
And, as long as there's an API, I am fairly certain that maintaining a compat layer is a lot less work than retuning the LLM when the images change. (And you'll want to adjust your tuning, at least with current SOTA, or your error rate will reach Unpleasantville fairly quickly)
Yes, it seems easier on the face of it - but the reality of building an LLM pipeline will quickly point out a lot of edge cases.
Personally, I stopped using my Oura ring and don't have location history collected anywhere (that I control or know of). One of my big ideas for this is a native application for your phone and laptop that collects everything it can in the background and surfaces it somewhere you control (synced via a service you choose, lands on a machine you own). Maybe overkill for many people but being able to access that without giving it to another service would be something I'd use.
If there was a way to add a meta-prompt to Windows Recall like "Create a log entry every time I watch something with its title and URL" it could serve as a history whether things were watched on YouTube, Vimeo, or any other site, without requiring plugging into each service individually. Repeat ad nauseum for each thing to be logged, or perhaps someone can come up with a more clever query than I that catches everything sufficiently.
The level of granularity on many services might be surprisingly large, preventing introspection of the data at a useful level.