The key insight from the study is that even the users that did see an increase in productivity overestimated that increase.

Taken along with the dozens of other studies that show that humans are terrible at estimating how long it will take them to complete task, you should be very skeptical when someone says an LLM makes them x% more productive.

There’s no reason to think that the most skillful LLM users are not overestimating productivity benefits as well.