Re:Ask HN: What info do you web scrape for?
by cblock811 on Monday August 11, 2014 @01:22PM
(#8164173)
The hospitality and travel industries are very slow to update their technologies. I used to work with Ritz Carlton and St. Regis and even those brands are practically in the stone age, so I can't imagine how scraping for flight info would go.
I've thought of even building a simple event aggregator for some friends in the industry and they are blown away that it's possible. Then I remember how many venues are in cities like Charlotte and San Francisco and realize why these industries lag in technology. There just isn't a large pool of developers who want to solve their problems.
Do you have any projects you are currently working on?
Re:Ask HN: What info do you web scrape for?
by jerhinesmith on Monday August 11, 2014 @08:24PM
(#8166029)
Completely agreed on the "lag in technology" -- couple that with the fact that even if you do manage to book a flight as an affiliate, most providers only give you a flat fee (as opposed to hotel affiliates which tend to give a percentage).
Disclaimer: Those claims were relevant the last time I investigated (>1 year ago). It could have changed by now.
On the plus side, building out the flight tool really endeared me to hacking on things - which eventually lead me to leave my last job and co-found something new. :)
Re:Ask HN: What info do you web scrape for?
by robglas on Monday August 11, 2014 @01:01PM
(#8164071)
So, Skyscanner? (Pick'Whole month' / 'Whole year' for the date).
Re:Ask HN: What info do you web scrape for?
by jerhinesmith on Monday August 11, 2014 @08:21PM
(#8166017)
This is the first I've heard of Skyscanner, and they kind of get what I was going for, but I want more of a "set it and forget it" type of service -- something where I can enter in the closest airport and a list of cities I want to visit, and every time I log in I'll see the prices for those cities over the next several months (or get alerted via email, etc.).
Re:Ask HN: What info do you web scrape for?
by joshmlewis on Monday August 11, 2014 @08:29PM
(#8166043)
Sabre just opened their API to the public. Not sure if it's paid or not but we were going to do this exact idea for YC Hacks.
Re:Ask HN: What info do you web scrape for?
by jerhinesmith on Monday August 11, 2014 @10:20PM
(#8166352)
Oh wow, how did I miss that? I looked at sabre when I first started scraping, but the application process to just look at the APIs was exhausting.
Thank you!
Anyway... it turns out that flight APIs are ridiculously non-existent. I ended up scraping two different airline sites, but since it was against their terms, I never took the site any further.