I need to change it so it doesn't shuffle letters in an order too close to the solution. Maybe 3 letters in the same order is too similar?

Any of the eight tuples should probably not be in order ( or just let one of them through ). An easier method might be to limit shuffling to once every minute.

Neat concept!

edit: burningfrog's suggestion is an alternative to shuffle that would be even better https://news.ycombinator.com/item?id=30221479

I wouldn't bother, far too many people are capable of 'reading' the letters in arbitrary directions as in games such as Boggle. If you eliminate all orderings that include adjacent vertical, diagonal, zigzag, crossing, and reverse directions, you aren't going to be left with much.