Given the reaction from Cleavon Little I could fully buy that it was an ad-libbed line.

Then again, they are actors. It might have started as ad-libbed, but entirely possible it had multiple takes still to get it "just right".