Re:Bing is now Microsoft Bing as the search engine gets a re...
by dredmorbius on Tuesday October 06, 2020 @10:34AM
(#24698060)
Google search from Tor nodes generally gets CAPTCHA or rate-limit blocked regardless of whether you go through DDG or not. Google Web Search is effectively unusable via Tor.
I used to be able to do some interesting analysis through automated commandline searches on Google, looking at the estimated results count (see, e.g., https://old.reddit.com/r/dredmorbius/comments/3hp41w/trackin...), with some ... very creative rate limiting.
Because of how Google restricts information, that analysis took about 10k searches (1 each per term per site), conducted over a week or so.
Last I looked into extending those analyses that route was utterly broken --- non-JS, plain-HTML results simply aren't produced, let alone rate-limiting and bot detection. Scripted search doesn't contribute to ad views. (Though neither do adblocked ones.)
DDG by contrast has its 'lite‘ search. Unfortunately for my research uses, no result set counts.
There may be APIs for these or other services, I've not yet looked into that.
I used to be able to do some interesting analysis through automated commandline searches on Google, looking at the estimated results count (see, e.g., https://old.reddit.com/r/dredmorbius/comments/3hp41w/trackin...), with some ... very creative rate limiting.
Because of how Google restricts information, that analysis took about 10k searches (1 each per term per site), conducted over a week or so.
Last I looked into extending those analyses that route was utterly broken --- non-JS, plain-HTML results simply aren't produced, let alone rate-limiting and bot detection. Scripted search doesn't contribute to ad views. (Though neither do adblocked ones.)
DDG by contrast has its 'lite‘ search. Unfortunately for my research uses, no result set counts.
There may be APIs for these or other services, I've not yet looked into that.