In terms of experience, I’ve noticed that agents don’t always use skills the way you want; and I’ve noticed that they’re pretty good at browsing existing code and docs and figuring things out for themselves.
Is this an example of “the bitter lesson”? That’s conjecture, but I think pretty well-founded.
It could well be that specific formats for skills work better because the agents are trained on those specific formats. But if so, I think it’s just a local maximum.
On the other hand, the specific details of how to do things in YOUR software, in YOUR environment (especially if it's quirky) - that does beat asking the model to work it out from first principles each time.
I still prefer to rely on convention where possible - i.e, what's even better than writing a skill for say, "how to manage users in our ABC application, first call API a..." is to follow some widely established convention that means the obvious first thing the model would naturally try "just works".
Making your docs nice and modular, and having a high-level overview that tells you where to find more detailed info on specific topics, is definitely a good idea. We already know that when we're writing docs for human readers. The LLMs are already trained on a big corpus written by and for humans. There's no compelling reason why we need to do anything radically different to help them out. To the contrary, it's better not to do anything radically different, so that new LLM-assisted code and docs can be accessible to humans too.
Well-written docs already play nicely with LLM context.