The "no FFN for retrieval tasks" observation is interesting essentially arguing that if the knowledge is in the context, FFN weights are redundant for that task. Curious whether this generalizes to multi-turn tool calling where the model needs to track state across several calls, or if it breaks down there. Single-shot is the easy case.