The granularity is currently hard-coded (e.g. 512 tokens default chunk size, augmented with passes of 128 tokens). I have not heard of the idea of Frames, but it's interesting. Thanks for sharing - I'll probably do a deep dive sometime this weekend.
I say this with a hint of irony, knowing that I’m just some kid and you’re launching an awesome AI product right now, but I highly recommend looking back at mainstream ai from before it all started actually working, namely the classic book Artificial Intelligence: A Modern Approach. Just because they lacked the tools doesn’t mean there wasn’t serious, detailed thought put into what we should do in the technological situation we now find ourselves in. For that reason, this is (AFAIK) the main textbook taught in US grad schools for AI survey courses. Frames would be part of chapter 10, I’m guessing.
Will definitely be checking it out, Dan or no :)
Just curious, is the granularity able to itself be controlled by LLMs, or does that refer to traditional content-blind sliding windows? I’m big on Minsky’s idea of Frames[1] which emphasize understanding the same input in multiple instrumental contexts/frameworks/backgrounds. Something you’ve thought of?[1] https://web.media.mit.edu/~minsky/papers/Frames/frames.html