Re:Fmllm: 4mb training data, 100mb model, Fibonacci embeddin...
by gabriel666smith on Friday August 22, 2025 @08:05AM
(#44983494)
Thank you, I agree, I think it'd be helpful to publish aspects of it.
> Are you saying the training task is to ask for the (fib_i)th token rather than the next token?
Yes, functionally - I explained in more detail in another comment.
I'm not sure which is the key point (sort of what I'm trying to work out), but I'll get the model-generation code into the repo. Is that the best thing for you?
> Are you saying the training task is to ask for the (fib_i)th token rather than the next token?
Yes, functionally - I explained in more detail in another comment.
I'm not sure which is the key point (sort of what I'm trying to work out), but I'll get the model-generation code into the repo. Is that the best thing for you?