The time goes into calculating a fullscreen pixels-based effect (so yeah; buffer contents), the point of this library is not the FB implementation which is a simple / bare-metal one.

The point is to render graphics content in a consistent way across multiple CPU cores but still remain lightweight enough to be used with your own implementation, this might change but that was the initial goal.