I've been using https://github.com/brodie/cram for this. It's a neat little shell testing tool that can be told to check that every 4-space indented markdown code block output what it says it outputs, so I just cram my README.md.

An example of this in action: https://github.com/liskin/liscopridge/blame/68a656b7beb10a5c..., https://github.com/liskin/liscopridge/blob/68a656b7beb10a5cd...