News for hackers. Stuff that matters.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by wyager on Sunday April 17, 2022 @12:20PM (#31061960)
DDG has been de-listing a ton of stuff recently. They recently de-listed rdrama, a reddit-trolling website that came up on HN last year. What, exactly, do they think people use them for? If I wanted "result curation" or whatever euphemism for censorship, I would just use Google.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by kevin_thibedeau on Sunday April 17, 2022 @12:42PM (#31062141)
They've removed Yandex and are now at the mercy of Bing's censorship.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by BurdensomeCount on Sunday April 17, 2022 @03:37PM (#31063681)
Amazing that nowadays if you want the best "uncensored" results you have to go to the Russian Yandex (at least on all topics unrelated to Russia). How did our society get to such a point...
Re:“YouTube-dl” and “Pirate Bay” back on DDG by krick on Monday April 18, 2022 @12:23AM (#31067411)
It has been like that for the long time, actually. I've seen examples where somewhat political search clause (in Russian) yields completely different results on Google & Yandex (it's actually hard to tell which one "tweaks" the results, and what tweaking even really means, when you think about it — my guess would be both, but differently). But in a sense of intentionally hiding something "neutral" that I know perfectly well must be there, but isn't (and it seems like you can guess why) — I've never actually noticed in on Yandex, while on Google it has been like that for the whole decade already. (I don't find is that surprising, really. To me, it's perfectly logical, but I won't elaborate, because it would be political and extremely unpopular on HN, which seems like a worthless flamewar topic to me.)

But in my experience Yandex just isn't as good at searching things in English, as DDG/Bing. At least, I always preferred them for most use-cases except from some really specific ones. So it's a pity that this thing with DDG happened. I guess, I'll still continue to use it for now, because I don't want to hurt my productivity other that thing (and switching to a search engine I would have to adapt to most certainly would do that). But we'll see.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by GoblinSlayer on Monday April 18, 2022 @03:24AM (#31068239)
I suppose that's a real use case for a meta search engine: send queries to different search engines, then interlace the results removing duplicates.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by BurdensomeCount on Monday April 18, 2022 @05:58AM (#31068954)
DDG was supposed to be like this originally, but they stopped using Yandex a while ago so now its just a wrapper on Bing...
Re:“YouTube-dl” and “Pirate Bay” back on DDG by samoa42 on Monday April 18, 2022 @10:52AM (#31071460)
i mean yes, it's suboptimal. but hardly news that for the real deal, you gotta leave the main road.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by markdown on Sunday April 17, 2022 @03:48PM (#31063799)
If that isn't a rhetorical question... capitalism.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by kenoph on Sunday April 17, 2022 @05:07PM (#31064522)
Re:“YouTube-dl” and “Pirate Bay” back on DDG by mpalczewski on Sunday April 17, 2022 @05:03PM (#31064495)
> capitalism

If capitalism caused censorship you would expect the least capitalist places to have the least censorship but the opposite is true.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by kenoph on Sunday April 17, 2022 @05:14PM (#31064586)
If capitalism causes censorship, that doesn't mean it's necessarily the worst at causing censorship.

Additionally, we don't live in a stationary society, so, whatever capitalism did or did not cause in the past might not apply as-is today.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by stjohnswarts on Sunday April 17, 2022 @06:17PM (#31065032)
Capitalism (as practiced by democracies) allows you to start an alternative site that doesn't censor results as well. Totalitarianism (Communism) would not have. Choose your poison from amongst the various governments that have succeeded (for a while) in the past as I don't think Utopia is a possibility with humans as messed up as we are.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by markdown on Monday April 18, 2022 @07:13AM (#31069311)
I'm not sure what your point is. The GP commenter asked what brought us here... it was capitalism. It was people starting other sites that didn't censor or didn't have ads or didn't do this or that. The winners of capitalism are what we have now.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by samoa42 on Monday April 18, 2022 @10:49AM (#31071400)
right, russians went from cyber-shady to cyber-pure-evil and now everyone has to cut ties...
Re:“YouTube-dl” and “Pirate Bay” back on DDG by Firmwarrior on Sunday April 17, 2022 @01:34PM (#31062572)
Man, that's interesting, they removed rdrama.net but left KiwiFarms up, and now the main result for "rdrama" is the KiwiFarms thread about it
Re:“YouTube-dl” and “Pirate Bay” back on DDG by ldiracdelta on Sunday April 17, 2022 @12:44PM (#31062163)
They've "memory-holed", 1984-style, other conservative sites as well. Just another narrative-enforcing search-engine.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by eloisius on Monday April 18, 2022 @02:21AM (#31067964)
Regardless of whether you support or oppose this kind of content, there are perfectly good reasons search for it and share it. This whole "don't talk about it, don't look at it" strategy that tech companies are taking is downright scary. I support identifying and transparently outing state actors. I'm not comfortable with it, but even censoring or caveating content based on author or institution is better than censoring based on the content itself.

For example, someone close to me expressed support for the "Proud Boys." This person in particular has been duped by evangelical movements in the past that all have one curious detail in common: they prohibit masturbation. Half to make fun of him, half to help him, I wanted to share a link with him to the Proud Boy's official website because he wouldn't believe me when I said they ban jackin' it. Facebook Messenger (in private DMs) refused to send a message including the link. "Send failed. Operation could not be completed." I tried an archive.org archive of it, same thing.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by uuyi on Sunday April 17, 2022 @12:47PM (#31062185)
I prefer to say “shite removal” after the last half a decade.

At least TPB adds value to peoples lives.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by somenameforme on Sunday April 17, 2022 @01:10PM (#31062387)
It depends on what you want out of a search engine. Do you want a search that tries to give you what it thinks you want? Or do you want a search that functions more like a library index where you're going to get what you searched for, even if that might ironically not really be exactly what you wanted - so the onus of creating a more correct search term is on you.

This isn't a rhetorical question of course. There are major arguments for both. But I think a pretty good chunk of DDG users were more often after the library index than what ideally would be a librarian recommendation, but in reality is more like a stereotypical used car salesman style recommendation.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by fallingknife on Sunday April 17, 2022 @02:24PM (#31062984)
I would be fine with either. But what I really don't want is a search engine that gives me what it thinks I should want.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by colordrops on Sunday April 17, 2022 @01:58PM (#31062774)
The whole point is to have a place where you can see shite if you want. I don't need an online nanny.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by HeckFeck on Sunday April 17, 2022 @06:16PM (#31065018)
I might agree with shite. Or I might wish to read it for my own amusement. Or perhaps I want to argue against the shite, in which case, I need to know what shite is out there. In any case, it doesn't matter. I only need the search engine to be an intermediary, not a curator.

Besides, if someone accidentally searches for shite, he can revise his search terms. Just like how you might reword something if a listener was confused.

This "curation" just doesn't need to exist. But that's a moral argument and those applying censorship aren't moral, so won't be partial to it.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by ldiracdelta on Sunday April 17, 2022 @01:01PM (#31062317)
Yes, the Overton Window must be enforced. We have always been at war with East Asia.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by ta8903 on Sunday April 17, 2022 @01:15PM (#31062431)
Removal of TPB was a positive too. Can't imagine how much loss of revenue these pirate sites have caused over the years.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by Marsey on Sunday April 17, 2022 @03:33PM (#31063635)
Luckily searching "Marsey the Cat" on Bing still yields the correct knowyourmeme and Twitter pages.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by rideontime on Sunday April 17, 2022 @10:38PM (#31066716)
> If I wanted "result curation" or whatever euphemism for censorship, I would just use Google.

Both youtube-dl and the pirate bay are available on google.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by dmix on Sunday April 17, 2022 @07:17PM (#31065457)
Now we have to wait for DDG to manually fix their results every time Bing censors something? Great…

I wonder what Yandex is like these days…

Re:“YouTube-dl” and “Pirate Bay” back on DDG by asdff on Sunday April 17, 2022 @05:08PM (#31064537)
Isn't DDG just reskinned bing anyhow?
Re:“YouTube-dl” and “Pirate Bay” back on DDG by thaumasiotes on Sunday April 17, 2022 @10:04PM (#31066525)
Well, youtube-dl.org is still delisted from bing, so clearly something is different.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by scarburato on Sunday April 17, 2022 @03:12PM (#31063413)
Bing delisted rdrama too. I think it is related somehow (?)
Re:“YouTube-dl” and “Pirate Bay” back on DDG by mgdlbp on Sunday April 17, 2022 @04:06PM (#31063957)
I noticed some time ago that Google refuses to offer up Encyclopedia Dramatica unless you use the site: operator.

There was some drama in 2010 over ED being censored in Australia, but it looks like Google has since quietly delisted it completely.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by Beltalowda on Sunday April 17, 2022 @07:58PM (#31065751)
I once ended up on Encyclopedia Dramatica before I knew what it was. It was some semi-informational article about some person and then I scrolled down and there was an absolutely disgusting picture I'm not even going to describe here. I, for one, don't mind that it's not on the top of my search results.

The entire job of a search engine is to curate results from a vast internet and boil it down to what is probably interesting for you. You also don't want all kind of "shock sites" on top if you search for "gun homicide" or "ISIS beheadings". Most likely you want some background information on these things, not "shock sites". That's the service Google and such provide.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by wyager on Sunday April 17, 2022 @09:09PM (#31066229)
ED is an incredible repository of useful factual data dressed as a trolling/shock site. Google is absolutely not de-listing it because they're worried about you. There's much worse content available on google; the difference that the other stuff isn't politically sensitive.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by DoItToMe81 on Monday April 18, 2022 @08:05AM (#31069598)
ED is an amazing time capsule of the 4chan and SA adjacent cultures that sprouted up in the early 2000s. Sad to see the state it's in now.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by mgdlbp on Sunday April 17, 2022 @11:35PM (#31067092)
But don't you think a query containing the exact name of a website is fairly likely in search of content from that site? In terms of recognizing user intent, I'd consider the need to use a search operator here a failing.

I commented because learning of such special-casing of a site by Google [a] was quite memorable for me at the time; since I also see plenty of other threads around us debating search engine curation, what follows is

I came across some Reddit threads from a certain time in 2014 which noted that the top Google results for some subjects were ED pages that mentioned them. Perhaps ED's downranking was in response to that. Whatever the cause, this is actually a more severe downranking than that famously applied to thepiratebay.org [b]; for example, the query thepiratebay org foo returns only results from the actual site, [c] but the same query for ED returns no results from ED. [d]

I did find two more cases where Google does return a (single) page from ED. The sole query without any operators that does so is encyclopediadramatica online (and its punctuation/whitespace-equivalent variants), for which the Main Page is the first result and the only ED result. [e]

The other case is when searching for a phrase within quotes that occurs nowhere else in Google's index other than on ED. For example, the expected only result of the query "Encyclopedia Dramatica help pages" is the ED page containing that text. [f]

So, to be more precise, ED is severely downranked rather than delisted, albeit with the result that no one can find it on Google without already knowing its URL (or a hapax legomenon quoted from its pages).

-----

[a][b][c][d][e][f] If I may use a simplistic model of Google that determines a score for each potential search result (i.e., 'distinct' crawled URL) by starting with the same initial score for all results and applying a sequence of steps to each result that each change its score, the following is a speculative explanation for all these behaviours:

- Following the meat of the algorithm is the downrank-specific-sites step, which decreases the score of each page of a downranked site by some amount (affects both TPB and ED: TPB scores are decreased moderately to implement [b]; ED scores are decreased massively, explaining [a]).

- Then comes the query-contains-site-URL step, which greatly increases the scores of all pages of that site (TPB results are now higher than non-TPB results, explaining [c]; ED scores were decreased so much that they are still lower than all other results, explaining [d]).

- Then comes the query-is-site-URL-exactly step, which makes the score for that site's base URL (which in the case of ED redirects to the Main Page) higher than any other score (explaining [e]).

- Search operators are last, and thus the highest-priority (explaining [f] and the site: operator).

These steps have held up in general where applicable for all the queries that I've tried.

Re:“YouTube-dl” and “Pirate Bay” back on DDG by BurdensomeCount on Sunday April 17, 2022 @03:39PM (#31063700)
Yep, I think the delisting comes from Bing, not DDG since DDG just pulls all their results from Bing since they stopped also using Yandex.
Re:“YouTube-dl” and “Pirate Bay” back on DDG by leach on Sunday April 17, 2022 @09:45PM (#31066427)
I've started using Kagi myself
Re:“YouTube-dl” and “Pirate Bay” back on DDG by mastazi on Sunday April 17, 2022 @09:48PM (#31066445)
Question: I've seen Kagi and I'm very interested, is there a way to use it without creating an account? Is the account required due to it still being beta, or it will be required in the future as well?
Re:“YouTube-dl” and “Pirate Bay” back on DDG by Gareth321 on Monday April 18, 2022 @06:22AM (#31069058)
I think an account is required, for at least a couple of reasons.

1. They will eventually require payment.

2. The service provides a lot of customisation options. Up and down ranking domains, for example, and setting up special filters. This kind of customisation can be built over many years and should carry with the user between browsers and devices.

Personally, I'm less worried about privacy and more about receiving accurate and unbiased search results. Of course, you can always use a VPN and use a burner email when you create the account.