The zombie could believe it has free will the same way it holds any other belief - whether it happens to be a delusion is immaterial to whether it's able to hold the belief and there's nothing particularly unique about this particular belief. It's also far from implausible that belief in free will could have some evolutionary utility (or, more likely, be a concomitant of some other property that has evolutionary utility). None of this proves you wrong, of course, I just don't see why we should imbue our common sense intuitions in this one area with such authority when they have failed us in so many others.
I know that others will hold to the idea that they are and could be, but to me that renders us as zombies, and that isn't satisfactory in the sense that while a zombie that acts like a real person would be evolutionary fit, I see no reason why the zombie should believe it has free will, or could, and yet I do.