To expand on this idea: Place the crawler under a non-profit such as the Internet Archive, so the results can serve the public & the commercial agreements can fund their broader archival efforts too.