It will work for this use case. In any case, most probably a human will select from a bunch of generated texts. Whether the texts are generated by a copywriter or GPT-3 will be the difference. So for small texts like heading and CTA buttons this should work. Longer texts are a different story though.

More than A/B testing this might be a better fit for web site building tools like wix.com and and webflow.com