Given that the whole point is to perform script-like operations as an atomic unit in the kernel, it seems to me you'd be reinventing a lot of script-like stuff by exposing a "bare" API for generating a "program" to be run.

For example, consider what the API must support to make a channel program that lists all the child datasets of a given dataset, and takes a snapshot of each child dataset that has a certain ZFS property.

The looping cannot be done by the user program, as that would introduce edge cases when datasets are created or destroyed between listing and generating the snapshot commands.

As such, picking Lua as the API isn't a bad choice I think.