GuideStar is the veteran in this space. I agree that doing this based on web scrapes and robots.txt is probably going to be pretty tough to get quality results. GuideStar always sells there product on the premise that they're vetting the financial statements of non-profits of best results. The real money might be on figuring out a way to scale reading and classifying non-profit financials - then see if you can quality control using a set of patterns.