News for hackers. Stuff that matters.
Re:Brave Search beta by jonathansampson on Tuesday June 22, 2021 @12:55PM (#27594110)
You can use !g on Brave Search too, or turn on Fallback Mixing in Brave Search Settings, which will anonymously call out to Google and pull in results as needed. This helps to train the nascent engine more rapidly. I hope this helps!
Re:Brave Search beta by decrypt on Tuesday June 22, 2021 @01:23PM (#27594446)
I am not able to locate this setting. I am using Brave Search on Firefox. Is that available only on Brave Search on Brave browser?
Re:Brave Search beta by jonathansampson on Tuesday June 22, 2021 @01:47PM (#27594754)
Yes, the Fallback Mixing requires the Brave browser, since it pipes the request through the participating user's machine (only if the user has first opted-in to the feature).
Re:Brave Search beta by JeremyBanks on Tuesday June 22, 2021 @01:19PM (#27594403)
Is that legal?
Re:Brave Search beta by ocdtrekkie on Tuesday June 22, 2021 @01:39PM (#27594660)
Why wouldn't it be? Google scrapes the web to populate it's results, why wouldn't other search engines scrape the web as well? Google is a website.
Re:Brave Search beta by SamBam on Tuesday June 22, 2021 @05:50PM (#27597665)
It's not scraping static text in order to point you to those sites, it's using the features of the site to perform a service better than you can do yourself. It's completely different.

If I made a site that claimed to help you with your math homework and simply sent the queries to WolframAlpha, that would also not just be "scraping."

Re:Brave Search beta by ocdtrekkie on Tuesday June 22, 2021 @09:17PM (#27599576)
This is basically what Google does to Wikipedia and rebrands as "Knowledge Graph".
Re:Brave Search beta by SamBam on Tuesday June 22, 2021 @10:43PM (#27600207)
Google has donated many millions to the Wikimedia Foundation, basically in payment for this.

(But, again, there's a difference between scraping and echoing a request on another site and waiting for its response. The latter is basically unauthorized use of its API, not scraping.)

Re:Brave Search beta by ocdtrekkie on Wednesday June 23, 2021 @12:25AM (#27600781)
It'll be very telling if Google switches to using Wikipedia Enterprise: https://www.wired.com/story/wikipedia-finally-asking-big-tec...

My guess is Google's "donation" is pennies on the dollar from what they benefit from Wikipedia, and more of a token gesture than anything else.

Re:Brave Search beta by xpe on Tuesday June 22, 2021 @03:35PM (#27596147)
This is naive. Web sites have various terms of service.
Re:Brave Search beta by therein on Tuesday June 22, 2021 @04:03PM (#27596468)
Violating a website's ToS is hardly illegal, though.
Re:Brave Search beta by xpe on Wednesday June 23, 2021 @01:37AM (#27601115)
> Violating a website's ToS is hardly illegal, though.

Can you be more specific as to your claim? Not illegal in what sense(s)? And what is your basis for the claim?

I'm not a lawyer, but saying "violating a website's ToS is hardly illegal" is fraught advice.

While individuals may get some leeway when it comes to ToS violations (see [1] and [2]), I would expect companies scraping and/or extracting content would be treated differently.

[1]: https://www.eff.org/deeplinks/2010/07/court-violating-terms-...

[2]: https://arstechnica.com/tech-policy/2020/03/court-violating-...

[3]: https://www.octoparse.com/blog/10-myths-about-web-scraping

[4]: https://www.law.com/newyorklawjournal/almID/1202610687621/?s...

Re:Brave Search beta by Kiro on Wednesday June 23, 2021 @05:49AM (#27602633)
The only way for it to be illegal is if they're breaking a law, but then it would be illegal regardless of what the ToS says.
Re:Brave Search beta by lern_too_spel on Tuesday June 22, 2021 @03:43PM (#27596231)
Because Google's robots.txt disallows it, and those websites allow it.
Re:Brave Search beta by smsm42 on Tuesday June 22, 2021 @05:50PM (#27597671)
robots.txt is not a legal contract. It's just a convention to express the wishes of the site author, but there's no legal obligation to follow these wishes.
Re:Brave Search beta by lern_too_spel on Tuesday June 22, 2021 @06:30PM (#27598079)
It does indicate that those other sites want Google to scrape them, while Google does not want others to scrape their results, which is an important distinction ocdtrekkie ignored for whether the scrapee will want to take legal action.
Re:Brave Search beta by ocdtrekkie on Tuesday June 22, 2021 @09:18PM (#27599581)
You may wish to review https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-l...

Google Search results are definitely "public data" so long as Google provides them to anyone who asks.

Re:Brave Search beta by lern_too_spel on Tuesday June 22, 2021 @11:18PM (#27600403)
Then why does Startpage pay Google and DDG pay Microsoft?
Re:Brave Search beta by ocdtrekkie on Wednesday June 23, 2021 @12:29AM (#27600802)
While scraping search results isn't illegal, by any means, it's also not illegal for Google or Microsoft to block requests they believe are from competing search engines. Presumably the cost of paying them is less than the cost of hiring engineers to constantly try to find new ways to outwit Google and Microsoft engineers.

Again, if scraping data from websites without permission, Google simply wouldn't exist. Bear in mind, robots.txt is a feature that Google and Microsoft choose to respect, but the default assumption search engines have made from the beginning, is that they are free to grab whatever they want from the web, unless you ask them otherwise to please not.

Re:Brave Search beta by lern_too_spel on Wednesday June 23, 2021 @01:39AM (#27601121)
> the default assumption search engines have made from the beginning, is that they are free to grab whatever they want from the web, unless you ask them otherwise to please not.

Which Google's robots.txt does.

> scraping search results isn't illegal, by any means

While scraping the results for yourself to look at might be OK, scraping results to display verbatim in another search engine without permission stretches fair use.

Re:Brave Search beta by ocdtrekkie on Wednesday June 23, 2021 @08:08AM (#27603483)
> While scraping the results for yourself to look at might be OK, scraping results to display verbatim in another search engine without permission stretches fair use.

No, it doesn't, because Google results aren't copyrightable, hence, there is no such thing as fair use. It's just information anyone is free to collect and use as they see fit.

Re:Brave Search beta by lern_too_spel on Wednesday June 23, 2021 @10:34AM (#27605020)
Why would rankings not be copyrightable?
Re:Brave Search beta by ocdtrekkie on Wednesday June 23, 2021 @07:04PM (#27611676)
Why would they be? Again, if all things being copyrightable by default, Google could not even exist, they assume they have the right to consume any data they want.

If a monkey can't copyright a selfie because they're not a person, an algorithmically generated spew of stuff Google ripped from elsewhere certainly lacks merit for copyright.

Re:Brave Search beta by lern_too_spel on Thursday June 24, 2021 @03:34AM (#27614875)
All things are copyrighted by default. Once again, those websites grant a license to search engines to consume their content via robots.txt, and Google does not.
Re:Brave Search beta by mthoms on Tuesday June 22, 2021 @01:46PM (#27594744)
It's just a redirect to Google.
Re:Brave Search beta by decrypt on Tuesday June 22, 2021 @01:58PM (#27594893)
There seem to be two different features:

1. Redirection to Google.

2. Piping Google's results back to Brave Search's UI: https://news.ycombinator.com/item?id=27594754

The original commenter was asking about the latter.

Re:Brave Search beta by mthoms on Tuesday June 22, 2021 @02:29PM (#27595327)
Ah, I see, Thanks for the heads up.
Re:Brave Search beta by Siira on Wednesday June 23, 2021 @02:06AM (#27601299)
Is scraping Google for your own commercial search engine legal?