News for hackers. Stuff that matters.
Re:How we scrape 300k prices per day from Google Flights by namdnay on Saturday June 13, 2020 @02:46PM (#23511652)
Google Flights isn’t a scraper, it’s an evolution of ITA matrix from what I remember, directly connected to that GDS. They aren’t piggy backing on someone else’s servers.

Which is what this guy could have done, instead of behaving like pond scum. It’s not like it’s particularly complicated to get programmatic access to a GDS API, that’s what they’re there for.

Re:How we scrape 300k prices per day from Google Flights by bryanrasmussen on Saturday June 13, 2020 @03:35PM (#23512046)
Pond scum? He's scraping some data from a company that got rich scraping data and that probably will tell him to stop doing it. I mean if he's pond scum, what level of scum are those guys with upshot sites? What level of scum is Mark Zuckerberg? Pond scum is generally supposed to be pretty scummy, I can think of thousands of people more scummy than someone scraping from google.
Re:How we scrape 300k prices per day from Google Flights by gusgordon on Saturday June 13, 2020 @02:55PM (#23511722)
It’s expensive to get access to a GDS API and, from what I’ve heard, the data they provide is quite difficult to work with. There’s a reason Google bought ITA for $700m, right? If this project ever grows, it could make sense to pull from a GDS.
Re:How we scrape 300k prices per day from Google Flights by namdnay on Saturday June 13, 2020 @03:05PM (#23511813)
> It’s expensive to get access to a GDS API and, from what I’ve heard, the data they provide is quite difficult to work with.

Well, it’s expensive to provide live answers for flight search queries across hundreds of airlines and thousands of airports... Some of the old booking interfaces are ugly, but for simple searching most of them provide relatively sane REST/JSON

I don’t understand your attitude, steal it until you make it?

Re:How we scrape 300k prices per day from Google Flights by james412 on Saturday June 13, 2020 @03:31PM (#23512010)
That's the attitude of just about every successful company in history. Once large enough, some of them (e.g. YouTube) even force industrial changes to accommodate all the theft that made them successful.

Meanwhile on the topic of attitudes, referring to a startup as 'pond scum' simply because they scrape an extremely expensive data set, especially regarding an industry with a long and controversial history of strategies designed to avoid price transparency.. hmm.

Re:How we scrape 300k prices per day from Google Flights by dlhavema on Saturday June 13, 2020 @02:52PM (#23511697)
Cool. I did not know that. Thanks for the clarification.