If you restrict the search space of the input, you also reduce the possible outputs, and that makes it easier to narrow down the key. However, it's only really plausible on very small inputs, another channel of attack (e.g. the ones illustrated in the article), or if you know something specific about the input.
TL;DR don't base64 without a reason because it gives the attacker more bytes with fewer values to work with.
This sounds scary and please tell me more. In particular what happens in the pathological case where the input consists of 32/64/128/256 bit blocks, and in each block all bits are zero except the last one which may be one or zero?