You only need ~4k words for 2 bytes-per-word alphanumeric encoding (keyspaceLength^bytesPerWord). There is a "special" dictionary identifier prefix and a padding indicator postfix, however (mostly for dev purposes).

Scrypt would probably be fine, though the addition of a secret salt (wouldn't need entropy, just secrecy) would be much better. On reflection, XOR would be stupid bad :B