News for hackers. Stuff that matters.
Re:Brave Search beta by mrpf1ster on Tuesday June 22, 2021 @12:44PM (#27593970)
DuckDuckGo allows you to do a Google search by prepending "!g" to any query. So usually I do that for the last 5% of queries that DDG fails on.
Re:Brave Search beta by jonathansampson on Tuesday June 22, 2021 @12:55PM (#27594110)
You can use !g on Brave Search too, or turn on Fallback Mixing in Brave Search Settings, which will anonymously call out to Google and pull in results as needed. This helps to train the nascent engine more rapidly. I hope this helps!
Re:Brave Search beta by decrypt on Tuesday June 22, 2021 @01:23PM (#27594446)
I am not able to locate this setting. I am using Brave Search on Firefox. Is that available only on Brave Search on Brave browser?
Re:Brave Search beta by jonathansampson on Tuesday June 22, 2021 @01:47PM (#27594754)
Yes, the Fallback Mixing requires the Brave browser, since it pipes the request through the participating user's machine (only if the user has first opted-in to the feature).
Re:Brave Search beta by JeremyBanks on Tuesday June 22, 2021 @01:19PM (#27594403)
Is that legal?
Re:Brave Search beta by ocdtrekkie on Tuesday June 22, 2021 @01:39PM (#27594660)
Why wouldn't it be? Google scrapes the web to populate it's results, why wouldn't other search engines scrape the web as well? Google is a website.
Re:Brave Search beta by SamBam on Tuesday June 22, 2021 @05:50PM (#27597665)
It's not scraping static text in order to point you to those sites, it's using the features of the site to perform a service better than you can do yourself. It's completely different.

If I made a site that claimed to help you with your math homework and simply sent the queries to WolframAlpha, that would also not just be "scraping."

Re:Brave Search beta by ocdtrekkie on Tuesday June 22, 2021 @09:17PM (#27599576)
This is basically what Google does to Wikipedia and rebrands as "Knowledge Graph".
Re:Brave Search beta by SamBam on Tuesday June 22, 2021 @10:43PM (#27600207)
Google has donated many millions to the Wikimedia Foundation, basically in payment for this.

(But, again, there's a difference between scraping and echoing a request on another site and waiting for its response. The latter is basically unauthorized use of its API, not scraping.)

Re:Brave Search beta by ocdtrekkie on Wednesday June 23, 2021 @12:25AM (#27600781)
It'll be very telling if Google switches to using Wikipedia Enterprise: https://www.wired.com/story/wikipedia-finally-asking-big-tec...

My guess is Google's "donation" is pennies on the dollar from what they benefit from Wikipedia, and more of a token gesture than anything else.

Re:Brave Search beta by xpe on Tuesday June 22, 2021 @03:35PM (#27596147)
This is naive. Web sites have various terms of service.
Re:Brave Search beta by therein on Tuesday June 22, 2021 @04:03PM (#27596468)
Violating a website's ToS is hardly illegal, though.
Re:Brave Search beta by xpe on Wednesday June 23, 2021 @01:37AM (#27601115)
> Violating a website's ToS is hardly illegal, though.

Can you be more specific as to your claim? Not illegal in what sense(s)? And what is your basis for the claim?

I'm not a lawyer, but saying "violating a website's ToS is hardly illegal" is fraught advice.

While individuals may get some leeway when it comes to ToS violations (see [1] and [2]), I would expect companies scraping and/or extracting content would be treated differently.

[1]: https://www.eff.org/deeplinks/2010/07/court-violating-terms-...

[2]: https://arstechnica.com/tech-policy/2020/03/court-violating-...

[3]: https://www.octoparse.com/blog/10-myths-about-web-scraping

[4]: https://www.law.com/newyorklawjournal/almID/1202610687621/?s...

Re:Brave Search beta by Kiro on Wednesday June 23, 2021 @05:49AM (#27602633)
The only way for it to be illegal is if they're breaking a law, but then it would be illegal regardless of what the ToS says.
Re:Brave Search beta by lern_too_spel on Tuesday June 22, 2021 @03:43PM (#27596231)
Because Google's robots.txt disallows it, and those websites allow it.
Re:Brave Search beta by smsm42 on Tuesday June 22, 2021 @05:50PM (#27597671)
robots.txt is not a legal contract. It's just a convention to express the wishes of the site author, but there's no legal obligation to follow these wishes.
Re:Brave Search beta by lern_too_spel on Tuesday June 22, 2021 @06:30PM (#27598079)
It does indicate that those other sites want Google to scrape them, while Google does not want others to scrape their results, which is an important distinction ocdtrekkie ignored for whether the scrapee will want to take legal action.
Re:Brave Search beta by ocdtrekkie on Tuesday June 22, 2021 @09:18PM (#27599581)
You may wish to review https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-l...

Google Search results are definitely "public data" so long as Google provides them to anyone who asks.

Re:Brave Search beta by lern_too_spel on Tuesday June 22, 2021 @11:18PM (#27600403)
Then why does Startpage pay Google and DDG pay Microsoft?
Re:Brave Search beta by ocdtrekkie on Wednesday June 23, 2021 @12:29AM (#27600802)
While scraping search results isn't illegal, by any means, it's also not illegal for Google or Microsoft to block requests they believe are from competing search engines. Presumably the cost of paying them is less than the cost of hiring engineers to constantly try to find new ways to outwit Google and Microsoft engineers.

Again, if scraping data from websites without permission, Google simply wouldn't exist. Bear in mind, robots.txt is a feature that Google and Microsoft choose to respect, but the default assumption search engines have made from the beginning, is that they are free to grab whatever they want from the web, unless you ask them otherwise to please not.

Re:Brave Search beta by lern_too_spel on Wednesday June 23, 2021 @01:39AM (#27601121)
> the default assumption search engines have made from the beginning, is that they are free to grab whatever they want from the web, unless you ask them otherwise to please not.

Which Google's robots.txt does.

> scraping search results isn't illegal, by any means

While scraping the results for yourself to look at might be OK, scraping results to display verbatim in another search engine without permission stretches fair use.

Re:Brave Search beta by ocdtrekkie on Wednesday June 23, 2021 @08:08AM (#27603483)
> While scraping the results for yourself to look at might be OK, scraping results to display verbatim in another search engine without permission stretches fair use.

No, it doesn't, because Google results aren't copyrightable, hence, there is no such thing as fair use. It's just information anyone is free to collect and use as they see fit.

Re:Brave Search beta by lern_too_spel on Wednesday June 23, 2021 @10:34AM (#27605020)
Why would rankings not be copyrightable?
Re:Brave Search beta by ocdtrekkie on Wednesday June 23, 2021 @07:04PM (#27611676)
Why would they be? Again, if all things being copyrightable by default, Google could not even exist, they assume they have the right to consume any data they want.

If a monkey can't copyright a selfie because they're not a person, an algorithmically generated spew of stuff Google ripped from elsewhere certainly lacks merit for copyright.

Re:Brave Search beta by lern_too_spel on Thursday June 24, 2021 @03:34AM (#27614875)
All things are copyrighted by default. Once again, those websites grant a license to search engines to consume their content via robots.txt, and Google does not.
Re:Brave Search beta by mthoms on Tuesday June 22, 2021 @01:46PM (#27594744)
It's just a redirect to Google.
Re:Brave Search beta by decrypt on Tuesday June 22, 2021 @01:58PM (#27594893)
There seem to be two different features:

1. Redirection to Google.

2. Piping Google's results back to Brave Search's UI: https://news.ycombinator.com/item?id=27594754

The original commenter was asking about the latter.

Re:Brave Search beta by mthoms on Tuesday June 22, 2021 @02:29PM (#27595327)
Ah, I see, Thanks for the heads up.
Re:Brave Search beta by Siira on Wednesday June 23, 2021 @02:06AM (#27601299)
Is scraping Google for your own commercial search engine legal?
Re:Brave Search beta by zzyzxd on Tuesday June 22, 2021 @12:54PM (#27594087)
The frustration I am talking about is:

Check query result -> realize the result is bad -> scroll back to the search bar -> place cursor at the beginning of the query -> enter "!g" -> redirect to Google.com

It is not too bad on desktop, but doing it once on a smartphone is more than enough to push me to switch back to Google.

Re:Brave Search beta by amoshi on Tuesday June 22, 2021 @01:01PM (#27594165)
The !g can be almost anywhere in the query, it just can't be followed by non-whitespace or prefixed by special signs.
Re:Brave Search beta by freedomben on Tuesday June 22, 2021 @02:35PM (#27595389)
This is what I wanted to say. It's a life saver on mobile especially. Just append !g to the query (or !gi for google images, !gm for maps, etc)
Re:Brave Search beta by the-pigeon on Tuesday June 22, 2021 @12:56PM (#27594114)
Personally I find it pretty predictable which queries duck duck go will fail on. Basically very niche ones.

So I just prepend based on what I'm searching for to begin with instead of after a failure. But I've also been using duckduckgo as my default for over a decade so I've gotten used to it.

Re:Brave Search beta by gnull on Tuesday June 22, 2021 @01:01PM (#27594168)
If you use Tridactyl, you can quickly jump to the search bar with “gi”. That's what I do.
Re:Brave Search beta by tonyspiff on Tuesday June 22, 2021 @05:45PM (#27597622)
you can easily create a command for it (and bind to key(s)):

  command ddgGoogleBang composite js (new URLSearchParams(window.location.search)).get('q') + ' !g' | urlmodify_js -q q | open
Re:Brave Search beta by windowojji on Wednesday June 23, 2021 @12:53AM (#27600927)
I'm amazed that DDG/Brave don't understand that typing something immediately after a "!" on a smartphone is annoying and difficult. DDG only supports "!" after the character for a small subset of bangs. Thankfully "g!" is one of them, but it's immensely easier to type on smartphone.
Re:Brave Search beta by smithza on Tuesday June 22, 2021 @02:42PM (#27595457)
most browsers support <c-l> to hop the cursor to the url bar for searches or url entries
Re:Brave Search beta by fictorial on Tuesday June 22, 2021 @01:27PM (#27594489)

    / <c-e> !g <enter>
Re:Brave Search beta by unicornporn on Tuesday June 22, 2021 @02:47PM (#27595534)
Or you could just use https://startpage.com/ and get Google quality results 100% of the time.
Re:Brave Search beta by mackrevinack on Wednesday June 23, 2021 @06:52AM (#27603000)
that's also 100% reliant on google though. what would happen if that became a lot more popular than it is now? would google try to sabotage it in any way?

i would rather support and spread the word about search engines that don't rely solely on google

Re:Brave Search beta by unicornporn on Friday June 25, 2021 @12:59AM (#27627292)
> that's also 100% reliant on google though. what would happen if that became a lot more popular than it is now? would google try to sabotage it in any way?

Yes, same applies to DDG/Bing though.

Re:Brave Search beta by aazaa on Tuesday June 22, 2021 @01:47PM (#27594756)
It redirects to Google. So the effect (including tracking) is identical to just doing a Google search.
Re:Brave Search beta by jonathansampson on Tuesday June 22, 2021 @02:13PM (#27595095)
This is incorrect. Fallback-Mixing, if you have enabled it (which requires Brave), issues an anonymous query to Google, lacking any cookies or other persistent state for that domain. These results are then presented along with Brave Search's own results. There's no tracking involved. If you perform a direct Google search, you're passing along your cookies as well.
Re:Brave Search beta by Semaphor on Tuesday June 22, 2021 @02:45PM (#27595492)
The user was replying to using DDG with !g

In an indirect way, it’s possible by doing !sp on DDG, which redirects the search to startpage, which shows untracked google results.

Re:Brave Search beta by jonathansampson on Tuesday June 22, 2021 @08:00PM (#27598983)
Ah, my apologies for any confusion. For what it's worth, the !sp bang is supported on Brave Search as well :)
Re:Brave Search beta by mastazi on Tuesday June 22, 2021 @06:32PM (#27598095)
As I said in my other comment[1], you might be interested in using !sp which gives you the same results without Google tracking

[1] https://news.ycombinator.com/item?id=27598042

Re:Brave Search beta by Dah00n on Tuesday June 22, 2021 @06:42PM (#27598231)
Startpage is owned by an adtech company now.
Re:Brave Search beta by mastazi on Tuesday June 22, 2021 @07:09PM (#27598495)
that doesn't change the fact that there are no trackers and the fact that it uses Google results
Re:Brave Search beta by Dah00n on Tuesday June 22, 2021 @07:35PM (#27598782)
No, but it does mean that you are likely to use it without knowing when they do start tracking you. Because of course an adtech company will (and of course they will deny until the day they change it).
Re:Brave Search beta by mastazi on Tuesday June 22, 2021 @07:45PM (#27598866)
> Because of course an adtech company will

then it seems they're being quite slow, since the acquisition happened almost 2 years ago.

Re:Brave Search beta by JohnWhigham on Wednesday June 23, 2021 @08:12AM (#27603513)
This is silly. 95% of what one can track on the front end can be done on the back end. If you hit Google's servers, you're being tracked. Period.