I haven’t read this paper but what you described is commonly done (look up top-k or top-p sampling and beam search as examples).