Re:Show HN: WrapAPI v2 – Build APIs, scrapers, bots on any w...
by wrapapi on Friday April 21, 2017 @07:37PM
(#14170279)
@OhSoHumble: this actually came to mind for us too in terms of building better experiences on top of common government services. I'd love to chat and see how I can help, so shoot me an email at [email protected]
Re:Show HN: WrapAPI v2 – Build APIs, scrapers, bots on any w...
by dhruvkar on Friday April 21, 2017 @07:34PM
(#14170267)
What was the vendor's website?
Re:Show HN: WrapAPI v2 – Build APIs, scrapers, bots on any w...
by OhSoHumble on Friday April 21, 2017 @10:16PM
(#14170787)
socialserve.com
An example of this is California drought data: automatically grabbing data on the drought is incredibly difficult because it involves scraping HTML tables. I tried to build an API that presents drought data so volunteers would have an easier time building out data visualizations. I ended up just getting exhausted doing all the scraping work.
I then moved onto a new project: building a free-to-use Padmapper for affordable housing. The data for income restricted apartment units are driven by a government contracted vendor. A city county will declare income stabilization policies and legally enforce them against landowners and then the landowners would send over their list of units to the vendor.
This would be great except the vendor does the bare minimum. Padmapper looks amazing but, really, it's only applicable for the upper middle class due to explosive housing costs in the Bay Area. So, in order to provide a more modern website and mobile application for the community, I started to scrape the vendor's website. It was terrible. I kept getting throttled. So I gave up.