Re:Avoiding bot detection: How to scrape the web without get...
by oxymoron on Monday November 01, 2021 @03:00AM
(#29063774)
Yeah, there’s a central service that all Flight search is connected to, irregardless of airline. The airlines are charged per search to that api, so they monitor their ”look to book” rationvery closely. That ratio remains quite stable im the absence of bots, but skyrockets with any bot activity. Hence, they know from that metric how big of a bot problem they have and how much money they are losing. Major flight search software vendors have dedicated teams for this.
I’ve noticed most sites won’t let you search business fares efficiently, so I made my own for Google Flights which only worked for like 6months until they added bunch of changes that made it near impossible to scrape.