If I had to do it again I would probably ditch the puppetmaster altogether and use an rsync server to distribute the entire configure repository to every server, then run puppet locally to apply changes. This way you can simply modify any local repository and run puppet to apply the desired configuration to any machine you want. When you're happy with the changes you can check them in.
Using the puppetmaster and the puppet fileserver was trickier, essentially I would use FACTER_var="value" to pass in a value to puppet that would use local files rather than central files (which came pretty close to the purely decentralized model anyway).
[Disclaimer: I work for Rails Machine.]
It's essentially a headless puppet that centers around a workflow of testing changes from an individual checkout of the puppet code on a target server, testing no-op applies of the manifests, and applying the manifest until you're happy enough to commit, push, and roll-out.
This won't help with your second annoyance, sadly, but it should definitely help with the first in quickly pinpointing these sorts of issues without having a messy commit history.
Also, regarding testing. I think this is an issue with both Chef and Puppet. Something I hope someone addresses at some point in the future. I've seen some custom tools with some promise (Chef focused) but perhaps a Vagrant setup might be the best answer these days.
https://github.com/apenwarr/redo/blob/master/Documentation/r...
Setting up puppet was relatively straight forward. We had the puppetd auto-updating our servers for a while, but ultimately decided to manually run it when deploying changes. Managing zero-downtime changes was more error prone with it running.
Some aspects of Puppet have over time proved frustrating to us. The top annoyance is we never quite figured out a good way to test our puppet changes before checking them into git to deploy them to our puppetmaster. That has lead to a number of "fixing errors" type commits. The second annoyance we've found is actually highlighted as a feature: no implicit ordering of operations. While it might sound great to be able to reorganize your configs without fear of breaking the deployment, we've found that the tradeoff is that you don't find out that your configuration doesn't define its dependencies correctly until you try to kick a new server after spending months incrementally adding to your existing servers. For us, at least, having an implicit top to bottom ordering would lessen that headache.
Despite some of these headaches, simply having our configuration in version control is a huge win for us. We can setup a box much more easily, and we have a comment trail of why changes were made.