I actually missed the built-in noise function and instead used the same library you linked to. I tried switching to built-in C++ implementation just now and it was actually slower.

The Lua library for simplex does some FFI trickery and avoids crossing Lua/C++ barrier. Mike Pall is credited in comments, so I guess I'm not surprised it's faster.