Two reasons: it's a bad idea and it wouldn't help very much. Each individual NTP server is only generating a modest amount of traffic and such a response might well go unnoticed by the NTP server. Also, it would mean we'd have to generate 400Gbps back to the NTP servers creating an enormous amount of traffic.
To actually get the benefits you're suggesting, CloudFlare would have to actually DDoS (spoof data from multiple IPs to the point of overwhelming the target) every participating NTP server. So on a 100Gbps attack, they'd need to send out, I dunno, 1Tbps of spoofed traffic? Not probably a wise idea.
The correct way is to just contact the network abuse team. That way I get an email telling me to fix my server. And it's taken care of as quickly as possible.
And the real correct way is for all ISPs to implement source filtering, but last I checked, that was laughably far from being implemented. Lots of ISPs would even take sources of private IPs and merrily forward them on.
This would have two benefits. Firstly the owner of the insecure NTP server is going to get a nasty message to fix their damn server, and secondly, the insecure NTP server gets taken out of the attack and becomes useless to the attacker.
As a visual reference, it would be a bit like in Star Wars when Mace Windu fights Palpatine on Coruscant [1]: http://www.youtube.com/watch?v=Pk4AiCnMqpg#t=2m35s
Eventually these server provider who have left their servers wide open will get the message when there NTP servers no longer respond?
1. http://starwars.wikia.com/wiki/Showdown_on_Coruscant