Ask YC: Any crawler experts out there?
Hi. Wondering if there are any crawler experts on here who can help me. We're wanting to create a crawler to visit some sites that have forms, lists of items and detail of items. They're all in the real estate market and we want to capture the properties and pull out the latest ones. I'm being told that we need to create a specific crawler for each site, but I was wondering if we could create...