I find writing a debug server easier than writing tests, but maybe that's just me. :) Especially for speculative things, the notion of whether the code is "correct" could change from moment to moment, so I find it much more useful to look at what it's doing than to test if what it's doing matches a spec that doesn't exist.

Hey, if it works, no problems here.

Personally, I've found it really helpful to be able to think of and test queries as pure functions (without side effects related to data storage). Composable, pure queries keep our code base really small, and allow us to code and test features in hours.

I end up writing and testing a lot of queries that I never use in production, but it's this play that allows the good stuff to bubble up.

I use Pig to write many of my Hadoop Mapreduce jobs and test my Pig script on a small dataset using Pig Grunt. Pig 0.9 has a lot more debug information as well.