I've been using Vagrant lately to spin up clean VMs to run Ansible against, and that's definitely something that you have to be very very careful about. I'm currently looking at setting up Jenkins to frequently re-provision from a clean VM, just to keep an eye on me to make sure I don't accidentally introduce a change that breaks things.

So far, I think the only real assumption my users have to deal with is that you need a full source tree checkout, not just the smaller part with the Ansible playbooks. The project uses Subversion, so it's quite easy to just checkout a subtree of the project instead of the whole thing. Which users did and immediately discovered how broken things are in that situation :)