Agree and I actually do the same thing. Follow up with an email summarizing the conversation and key points. Get a reply from the person acknowledging. Not bullet proof but definitely a leg to stand on.
As far as "legal test" things rarely make it to court (even with contracts). At the point that's happened you've already lost. They are normally settled before that for anything of a small nature. Anything larger (and it's hard to define small but I know it when I see it) needs a contract.
Not that I have had to put this to the legal test, but there is then this sort of written documentation, accessible to both parties, of what has been agreed and when. Having everything in an email folder is easier for me than keeping a log book.