News for hackers. Stuff that matters.
Re:Google No Longer Automatically Indexes Websites – WTF? by burnte on Monday June 05, 2023 @10:11AM (#36196413)
Another bug I'm noticing lately is it'll flat out ignore things sometimes, even if you put a term in quotes or try to exclude it with a -leadingdash. About 30% of the time if I use those operators, they'll have no effect on the results. I don't understand why they'd make things worse on purpose, but I don't know how it could be just a "mistake" no one noticed.
Re:Google No Longer Automatically Indexes Websites – WTF? by cmeacham98 on Monday June 05, 2023 @10:14AM (#36196461)
I notice this happening when the actual query would have returned 0 results, Google ""helpfully"" will modify your query (such as dropping quotes) to generate more results.

This is super annoying because it doesn't appear to inform you of this anywhere in the UI, until you click through to page 2 and see what it modified your query to be.

Re:Google No Longer Automatically Indexes Websites – WTF? by QuantumGood on Monday June 05, 2023 @10:39AM (#36196786)
Over the years, I've found the frequency of "0 result" queries has gone way up. Subjective anecdata from me, but it's a pretty big difference. There must be some large areas of their index that have been dropped over time.
Re:Google No Longer Automatically Indexes Websites – WTF? by morkalork on Monday June 05, 2023 @10:55AM (#36197047)
Also 1 page results. Around 10-20 links. Like, that's it? Really? No more? That's not what I see when I try the same query on Bing!
Re:Google No Longer Automatically Indexes Websites – WTF? by lispy4 on Monday June 05, 2023 @12:07PM (#36198394)
I've noticed the same. And sometimes you'll get search results showing 10+ pages, but if you actually follow through them, the results die by the second page. Google also omits many domains from search results now.
Re:Google No Longer Automatically Indexes Websites – WTF? by voidhorse on Tuesday June 06, 2023 @12:11AM (#36207989)
Which has the really annoying side effect that now you actually need to produce a worse query to get decent results.

A few years back I used to scoff at people who wrote searches in the form of literal questions instead of boiling things down to key terms, e.g. the "how do i..., what is the largest..." type searches.

Nowadays I not only need to write stupid searches like this to get better results but quite literally find that my brain has adapted the pattern and my past skill at crafting salient key term, operator driven queries is eroding.

Re:Google No Longer Automatically Indexes Websites – WTF? by winternett on Monday June 05, 2023 @10:25AM (#36196589)
Sites like Twitter and Instagram also frequently completely change the search term now to something else for certain queries. This practice is anti-competitive in the highest order. The very foundation of having a text search is to have an exact query match to begin with... The alternate spelling item should only be a suggestion in results at the most, but they've flipped this now, and that's outright deceptive.
Re:Google No Longer Automatically Indexes Websites – WTF? by jerf on Monday June 05, 2023 @01:16PM (#36199556)
It is located under the search query itself. For instance, see https://www.google.com/search?hl=en&q=%22spill%20clean%20big..., a nonsense query I made up with no hits. You should see

    No results found for "spill clean big search stain".

    Results for spill clean big search stain (without quotes):
on the page.

This isn't new, I've always seen Google doing something like this. It hasn't always been large on the page but it's always been there.

Re:Google No Longer Automatically Indexes Websites – WTF? by cmeacham98 on Monday June 05, 2023 @10:35PM (#36207363)
Might be desktop exclusive? For me on mobile (testing another random phrase as now yours hits this post), I don't see that text, or any other indication the query has been modified.
Re:Google No Longer Automatically Indexes Websites – WTF? by burnte on Tuesday June 06, 2023 @02:29PM (#36217101)
I'm used to that. But this is worse, this is when the search I enter would give the ideal results because I eventually contort the search parameters that I find some results.
Re:Google No Longer Automatically Indexes Websites – WTF? by chongli on Monday June 05, 2023 @10:42AM (#36196830)
Search engines in general have realized that it's more profitable to show you irrelevant results than to show you nothing. Furthermore, they've realized it's more profitable to show you irrelevant results laden with their ads than show you highly relevant results from ad-free sites.

Perverse incentives at work!

Re:Google No Longer Automatically Indexes Websites – WTF? by 20after4 on Monday June 05, 2023 @11:14AM (#36197369)
This is precisely what happened. When google merged with doubleclick.net the new company should have been named doubleclick.net and not google. The old google ceased to exist at that point and was swallowed by an advertising company.

I strongly agree with this bill hicks bit on advertising:

https://www.youtube.com/watch?v=-gd01vfKfr0

Re:Google No Longer Automatically Indexes Websites – WTF? by ceph_ on Monday June 05, 2023 @11:40AM (#36197894)
> When google merged with doubleclick.net the new company should have been named doubleclick.net and not google. The old google ceased to exist at that point and was swallowed by an advertising company.

This is one of the most unhinged takes I've read in a while.

Re:Google No Longer Automatically Indexes Websites – WTF? by formerly_proven on Monday June 05, 2023 @11:43AM (#36197941)
Regarding unhinged ideas, doubleclick is quite old, but is it old enough that opening a hyperlink would've typically required a double click at the time? Or is the metaphor here that their ads are so amazing people are double-clicking them in ecstasy?
Re:Google No Longer Automatically Indexes Websites – WTF? by Tagbert on Monday June 05, 2023 @12:37PM (#36198909)
Double-click, as others have said was never something you did with hyperlinks, even before the web.

Double-clicks were used with icons on the desktop because you could do more with an icon than just open it. You could move it, copy it, etc. Double-click was a convention for a shortcut to open the reference of the icon. A single-click would have not allowed those other actions.

Re:Google No Longer Automatically Indexes Websites – WTF? by Terretta on Monday June 05, 2023 @12:53PM (#36199209)
Because of this, double-click became business speak for going to the next level of detail, digging into, etc.

The idea behind this name for ads was: this company makes ads relevant and compelling, so users drill into them and find whatever you want to advertise.

For what it's worth, because of the affordance you mention, even though users didn't have to, they consistently double-clicked banner ads, and most things they wanted to activate, even after they learned they only had to single click the blue underlined things.

Re:Google No Longer Automatically Indexes Websites – WTF? by subpixel on Monday June 05, 2023 @09:12PM (#36206780)
Your use of the past-tense is premature, I see double-clicking all the time at work.
Re:Google No Longer Automatically Indexes Websites – WTF? by tempest_ on Monday June 05, 2023 @08:30PM (#36206448)
Have we moved so far past the desktop that this knowledge is starting to be lost?

I had to swap real quick to my desktop and see if I still had to double click an icon just to be sure.

I mostly access things with the super key / search now and I guess people with phones would just tap an icon.

Re:Google No Longer Automatically Indexes Websites – WTF? by emodendroket on Tuesday June 06, 2023 @01:27AM (#36208503)
You can configure single clicking in Windows, or at least you once could.
Re:Google No Longer Automatically Indexes Websites – WTF? by account42 on Tuesday June 06, 2023 @06:33AM (#36210662)
You can but that means hovering over files for a second will select them (and clear your previous selection). Other systems (e.g. KDE) manage single click to open without that annoyance.
Re:Google No Longer Automatically Indexes Websites – WTF? by account42 on Tuesday June 06, 2023 @06:31AM (#36210650)
> A single-click would have not allowed those other actions.

Yes it would. And did. Windows chose double click to open but other systems managed with a single click while still allowing you to drag around icons and files.

Re:Google No Longer Automatically Indexes Websites – WTF? by foobiekr on Monday June 05, 2023 @11:58AM (#36198240)
Opening a hyperlink has never required a double click in browsers. Not from Mosaic forward.
Re:Google No Longer Automatically Indexes Websites – WTF? by 20after4 on Monday June 05, 2023 @12:47PM (#36199083)
I was being hyperbolic but it's not that far from the reality of the situation. Google's decline started around the same time as that merger.

I'm not the only one who thinks this way: https://www.nytimes.com/2020/09/21/technology/google-doublec...

Re:Google No Longer Automatically Indexes Websites – WTF? by cduzz on Monday June 05, 2023 @01:37PM (#36199901)
See also Boeing merging McDonald Douglas[1] [1]https://qz.com/1776080/how-the-mcdonnell-douglas-boeing-merg...
Re:Google No Longer Automatically Indexes Websites – WTF? by hinkley on Monday June 05, 2023 @02:14PM (#36200546)
I know there were rumblings in the late 00's and early 10's about how McDonnell Douglas culture and executives were ruining Boeing.

But some people take a step farther back than this and blame Congress for the 737 MAX. They basically forced the merger, and unhappy weddings make for unhappy homes.

Re:Google No Longer Automatically Indexes Websites – WTF? by cduzz on Monday June 05, 2023 @10:00PM (#36207115)
I've seen plenty of mergers where there's a weird brain transplant and flippozambo! the acquired company's leadership is now in charge of the buying company. The fish, as they say, trots from the head.
Re:Google No Longer Automatically Indexes Websites – WTF? by stcroixx on Monday June 05, 2023 @09:03PM (#36206717)
I totally agree with you. Ads have influenced everything they’ve done since. It’s like a brilliant, talented individual who has been addicted to heroin for a decade.
Re:Google No Longer Automatically Indexes Websites – WTF? by eyegor on Monday June 05, 2023 @10:08PM (#36207165)
Yeah but google also became wildly successful to the point that they blow money on ventures with no real businesses plan and give up two years later when they can't turn a profit. They effectively have a blank check at all times. They're more like a businessman who's addicted to making money at the expense of all their personal relationships.
Re:Google No Longer Automatically Indexes Websites – WTF? by romusha on Monday June 05, 2023 @10:20PM (#36207247)
Is there any good search engine these days?
Re:Google No Longer Automatically Indexes Websites – WTF? by tomcatfish on Monday June 05, 2023 @11:51PM (#36207853)
grep
Re:Google No Longer Automatically Indexes Websites – WTF? by graevy on Tuesday June 06, 2023 @01:53AM (#36208686)
Not really. LLMs are good if you account for hallucination.
Re:Google No Longer Automatically Indexes Websites – WTF? by supriyo-biswas on Tuesday June 06, 2023 @04:58AM (#36210064)
phind.com

Kagi (don’t use it personally)

Re:Google No Longer Automatically Indexes Websites – WTF? by gramie on Monday June 05, 2023 @11:13AM (#36197350)
I find Amazon really irritating for that. I do a search for a very specific thing, and a ton of results always come back, often having nothing to do with my search request. And sponsored results both at the top and scattered through the results.
Re:Google No Longer Automatically Indexes Websites – WTF? by cptskippy on Monday June 05, 2023 @11:41AM (#36197903)
Amazon is gotten so bad that unless I know an exact part number or model then I don't bother. I'll go somewhere else for any research and only come back to Amazon if I want to price shop what I found.
Re:Google No Longer Automatically Indexes Websites – WTF? by user_666 on Monday June 05, 2023 @12:14PM (#36198488)
Amazon is so bad that I shop on Walmart's website now.
Re:Google No Longer Automatically Indexes Websites – WTF? by hex4def6 on Monday June 05, 2023 @12:53PM (#36199195)
Even with an exact part number, it will often push related items first. I was searching for a specific thermal printer, literally using the PN (something like C18647585), and it still decided to show me "sponsored" and related thermal printers first. So it somehow knew that part number as a keyword for thermal printers, but just didn't want to show me the one result that actually would be helpful (it was a third party seller, so maybe that penalizes the result?)
Re:Google No Longer Automatically Indexes Websites – WTF? by xoxxala on Monday June 05, 2023 @12:28PM (#36198743)
I get better results by searching Amazon via DDG, Brave or Kagi. Amazon's search, especially for books, is nearly useless by comparison.
Re:Google No Longer Automatically Indexes Websites – WTF? by tedmiston on Monday June 05, 2023 @07:30PM (#36205827)
Luckily for us we have the high-quality, independent book data provider Goodreads! /s
Re:Google No Longer Automatically Indexes Websites – WTF? by winrid on Monday June 05, 2023 @08:45PM (#36206592)
I found goodreads search to be quite good, where's it lacking for you?
Re:Google No Longer Automatically Indexes Websites – WTF? by efreak on Monday June 05, 2023 @09:53PM (#36207077)
Amazon owns Goodreads. It's not independent. It's also not mentioned on the site afaik. (They also own IMDb and a bunch of other internet companies that aren't Amazon-branded). If you want something independent, try storygraph or librarything.
Re:Google No Longer Automatically Indexes Websites – WTF? by winrid on Tuesday June 06, 2023 @12:18PM (#36215087)
Yeah I knew about IMDB. I know a guy at Amazon that said IMDB used to actually be in Perl, they rewrote it in Java over a few years.
Re:Google No Longer Automatically Indexes Websites – WTF? by tedmiston on Thursday June 08, 2023 @02:16PM (#36246252)
Goodreads is tolerable, but mainly as a data source. The product itself has been in maintenance mode for... a decade?... or basically since the Amazon acquisition.

They rolled out a completely new design semi-recently for... half of the pages... and left the other half on the 10+ year old styles.

It just feels like Amazon is happy to take advantage of its dominant position with Goodreads having a more complete catalog than any of the other more open offerings. And yet, they seem to invest no effort in modernizing or improving the site, making it more performant, etc. The moderation tools kinda suck too — doing super common things like merging (incorrect) duplicate listings is a PITA.

Also the app exhibits blatant conflicts of interest like prioritizing buying new books from Amazon over, e.g., digital library loans, with no option for users to configure that.

Nothing I'm saying here is new though - https://en.wikipedia.org/wiki/Goodreads#Criticism.

Re:Google No Longer Automatically Indexes Websites – WTF? by dredmorbius on Thursday June 08, 2023 @04:06AM (#36238691)
There's Worldcat, though its site revise last year made it useless for me.

I'm finding Open Library (part of the Internet Archive) is increasingly useful for book search.

You might also have success with a major library (e.g., British Library, Library of Congress, major US city libraries (NYC, Boston, LA, San Francisco, Chicago, etc.), and some academic libraries. Watch that these aren't in fact backed by Worldcat though. (Many local library systems are.)

Re:Google No Longer Automatically Indexes Websites – WTF? by kybernetikos on Monday June 05, 2023 @01:25PM (#36199687)
This is particularly bad if you search for a type of thing, e.g. "mechanical keyboard". Many of its top suggestions will be for nonmechanical keyboards and that won't be obvious without reading their descriptions carefully.
Re:Google No Longer Automatically Indexes Websites – WTF? by 0cf8612b2e1e on Monday June 05, 2023 @01:45PM (#36200065)
“Mechanical keyboard like”

What’s infuriating is how this lying has become normalized in “good” brands. For instance, try to buy a 60” TV. I do not think you can find one. They are all 59.5” and sold as ‘60” class’.

Re:Google No Longer Automatically Indexes Websites – WTF? by CamperBob2 on Monday June 05, 2023 @01:59PM (#36200291)
That usually means that discriminatory taxes or regulations are being dodged, for better or worse.
Re:Google No Longer Automatically Indexes Websites – WTF? by account42 on Tuesday June 06, 2023 @06:40AM (#36210709)
Perhaps that's one reason but it's also a >1% reduction in display area which is not nothing.
Re:Google No Longer Automatically Indexes Websites – WTF? by ajsnigrutin on Monday June 05, 2023 @07:11PM (#36205582)
Aliexpress is the worst for this...

"keyboard non mechanical like cherry switch membrane touch rgb light clicky gaming blogging ergonomic xiaomi redmi arduino android ios windows laptop desktop computer tablet phone", and if you sort by price, the first one is $0.99.

Then you have 5 different colors of plasticky $15 keyboards and a usb card reader for $0.99 to choose.

Re:Google No Longer Automatically Indexes Websites – WTF? by quickthrowman on Monday June 05, 2023 @06:36PM (#36205097)
Stop buying from Amazon. I haven’t bought anything from them in years. There is nothing that Amazon offers for sale that you can’t find somewhere else, aside from maybe entertainment content that they produce.

Don’t reward bad behavior or they’ll keep doing it.

Re:Google No Longer Automatically Indexes Websites – WTF? by account42 on Tuesday June 06, 2023 @06:45AM (#36210750)
What you can find on amazon that you can't find elsewhere (at least here):

- All your regular non-food needs in one cart.

- No-hassle refunds if there is a problem with the order.

The first one is just convenience and I could do without. The second one is where most other stores fail. Or at least enough of them that I don't want to risk having to phone their hotline or pay for shipping back because they or their delivery contractor fucked up - or is fraudulently claiming they tried to deliver when they made no such attempt at the specified address and instead want you to waste your time picking up the package at a random location accross town.

Re:Google No Longer Automatically Indexes Websites – WTF? by dopidopHN on Monday June 05, 2023 @11:34AM (#36197793)
* * *
Re:Google No Longer Automatically Indexes Websites – WTF? by 20after4 on Monday June 05, 2023 @12:52PM (#36199183)
Generating money for google is not the only metric that matters for the users. The incentives are perverse from the perspective of everyone other than google executives and investors with significant google holdings.
Re:Google No Longer Automatically Indexes Websites – WTF? by ajmurmann on Monday June 05, 2023 @01:07PM (#36199433)
And we are seeing alternatives like Kagi pop up because of it.
Re:Google No Longer Automatically Indexes Websites – WTF? by CamperBob2 on Monday June 05, 2023 @02:01PM (#36200316)
Let's ask Infoseek about that.
Re:Google No Longer Automatically Indexes Websites – WTF? by ballenf on Monday June 05, 2023 @12:52PM (#36199191)
Also on the HN front page right now is an example of the price we pay when we prune "undesirable" websites from our search indices:

https://www.quantamagazine.org/sci-fi-writer-greg-egan-and-a...

I don't know if 4chan is included in the google index, but I've never gotten a 4chan result in any search I can recall.

Re:Google No Longer Automatically Indexes Websites – WTF? by tayo42 on Monday June 05, 2023 @01:04PM (#36199379)
why would you expect to get a 4chan page? none of that data is persistent? iirc google relies on links to the page, so that is impossible, plus the content rotates constantly when they drop off the last page
Re:Google No Longer Automatically Indexes Websites – WTF? by ipaddr on Monday June 05, 2023 @01:54PM (#36200213)
Because you get 4chan results in duckduckgo and yandex.
Re:Google No Longer Automatically Indexes Websites – WTF? by tayo42 on Monday June 05, 2023 @02:09PM (#36200460)
What's the query and result?

I still don't get how you expect it to work when the content rotates quickly and disappears

If you search 4 Chan on Google it does come up in the results and a safe search warning

Re:Google No Longer Automatically Indexes Websites – WTF? by chownie on Monday June 05, 2023 @03:46PM (#36202462)
Only a couple boards on 4chan update quickly, the vast majority contain threads which stand for months at a time.
Re:Google No Longer Automatically Indexes Websites – WTF? by jstarfish on Monday June 05, 2023 @02:56PM (#36201334)
Funny you say that. I got referred to the local AI models thread (/lmg/) on the technology board just the other day.
Re:Google No Longer Automatically Indexes Websites – WTF? by tnffhjiuddsg on Monday June 05, 2023 @01:46PM (#36200080)
https://tinyurl.com/43mkh68w reverses these incentives

Google will remain on top as long as Apple, Mozilla and Samsung take their bribes

Re:Google No Longer Automatically Indexes Websites – WTF? by gpvos on Monday June 05, 2023 @02:49PM (#36201167)
Your use of Tinyurl doesn't really shorten the URL. Please don't obfuscate unnecessarily.
Re:Google No Longer Automatically Indexes Websites – WTF? by tnffhjiuddsg on Monday June 05, 2023 @03:01PM (#36201489)
Ideally, I'd like to delete messages after a day, a month, a year. But HN messages stay online forever and search engines eventually pick up on them

I just don't like being in the panopticon

Re:Google No Longer Automatically Indexes Websites – WTF? by throwawaymobule on Monday June 05, 2023 @03:13PM (#36201851)
that short URL goes to aisearch.vip
Re:Google No Longer Automatically Indexes Websites – WTF? by anticrymactic on Tuesday June 06, 2023 @02:45AM (#36209040)
Link is https://www.aisearch.vip/aisearch#

There are enough dead "short" links on forums already.

Re:Google No Longer Automatically Indexes Websites – WTF? by mech422 on Monday June 05, 2023 @10:33AM (#36196694)
I've noticed this with the quotes as well...
Re:Google No Longer Automatically Indexes Websites – WTF? by freitzkriesler2 on Monday June 05, 2023 @10:22AM (#36196557)
There was a Google search engineer on Reddit who claimed the opposite personally, Google is going down the trash but the alternatives aren't any better. Of course I can find it now, thanks Google.

I wish there was a search engine that ran like mid 2000s google but with a social media component so you can down vote SEO spammer blogs into oblivion.

Re:Google No Longer Automatically Indexes Websites – WTF? by lopis on Monday June 05, 2023 @10:29AM (#36196646)
Unfortunately, content farms can push new websites and blogs faster than you could ever downvote them. LLM are going to make that task increasingly easier. I've no idea how we're ever going to be able to search anything anymore using classic search engines. We either go back to website directories, or forward to AI generated content..
Re:Google No Longer Automatically Indexes Websites – WTF? by RajT88 on Monday June 05, 2023 @10:55AM (#36197043)
Web rings.

Think about it - some human element of trust and vouching for someone being added to the ring.

Re:Google No Longer Automatically Indexes Websites – WTF? by ddalex on Monday June 05, 2023 @11:14AM (#36197368)
Till you find out that most human will sell out ring links for a bit of cash with no problems.
Re:Google No Longer Automatically Indexes Websites – WTF? by RajT88 on Monday June 05, 2023 @11:42AM (#36197933)
Sure. It's a problem with peer-reviewed science journals even. There are no perfect solutions to monied interests bribing the curators.
Re:Google No Longer Automatically Indexes Websites – WTF? by mostlysimilar on Monday June 05, 2023 @11:38AM (#36197862)
Perhaps that is one additional layer of friction that will make human moderation / social voting feasible. The fire hose of AI trash content will come too rapidly for it to work at layer 1 (all content), but if the barrier to entry is a financial transaction to take over placement in a human-curated webring or directory it becomes easier to moderate / vote away the trash.
Re:Google No Longer Automatically Indexes Websites – WTF? by groby_b on Monday June 05, 2023 @12:04PM (#36198331)
Add a trust metric and chains of provenance. Bad ring link -> bad trust percolating up that chain. Little trust, your site isn't always shown as part of the ring. Too much loss of trust, you're out.

(Ultimately, this is a bad facsimile of human group behavior - all the way up to shunning people who deeply violate group norms. And I don't think it'll scale super-well. )

Re:Google No Longer Automatically Indexes Websites – WTF? by gregoryl on Monday June 05, 2023 @12:52PM (#36199180)
That's pagerank, right? The trust was built from href votes.
Re:Google No Longer Automatically Indexes Websites – WTF? by groby_b on Monday June 05, 2023 @02:40PM (#36200999)
Except there's no provenance or root of trust. There is (IIUC) no back-propagation of a penalty if sites violate trust, just an overall observational measure.

And I'd still say pagerank did work really well in an Internet where there was overwhelmingly trust. But in a world where default-trust is a bad stance, I believe there needs to be an equivalent of what "You can trust X" does in small in-person groups. (Or, alternatively "Sheesh, X went off and just destroyed all trust")

I do think it'll need to be more than a single metric, too. Trust is multidimensional by topic(E.g. "I trust the NYTs data science folks, I have zero trust for the OpEds"), and it is somewhat personal. (E.g. I might have experienced X lying to me, while they've been 100% honest to you - maybe in/outgroup, maybe political alignment, maybe differing beliefs, etc. Ultimately, what we call trust in an indirect situation is "most of my directly trusted folk vouch for that person)

Re:Google No Longer Automatically Indexes Websites – WTF? by nunuvit on Monday June 05, 2023 @11:43AM (#36197948)
Keyservers. You decide which keyservers to register with and to trust for verifying others. Browsers would handle en-decryption automatically and allow you to flag, filter, or finger (in the Unix sense).
Re:Google No Longer Automatically Indexes Websites – WTF? by CamperBob2 on Monday June 05, 2023 @02:02PM (#36200345)
No, they can't. Or at least, they don't. I see the same trash-fire sites on Google all the time. Google just DGAF.
Re:Google No Longer Automatically Indexes Websites – WTF? by dcow on Monday June 05, 2023 @10:29AM (#36196648)
Kagi is an alternative and it is worlds better. Try it out!

https://kagi.com

Re:Google No Longer Automatically Indexes Websites – WTF? by freitzkriesler2 on Monday June 05, 2023 @10:36AM (#36196741)
Checking this out, thanks mate.
Re:Google No Longer Automatically Indexes Websites – WTF? by rolisz on Monday June 05, 2023 @11:07AM (#36197242)
I'm also a huge fan of Kagi. I've been a paying user since they launched the paying subscription. Really happy with it!
Re:Google No Longer Automatically Indexes Websites – WTF? by dhc02 on Monday June 05, 2023 @11:11AM (#36197320)
Love kagi. The first time I got the "your payment was successful" notification I felt like I'd never get that much value out of it. But now, a few months later, I feel like I could never go back.
Re:Google No Longer Automatically Indexes Websites – WTF? by mordae on Monday June 05, 2023 @11:35AM (#36197802)
Though a subscriber myself, Kagi doesn't really add results, does it? It merely weeds out the trash for you. So you can get to the bottom of search results.
Re:Google No Longer Automatically Indexes Websites – WTF? by anonred on Monday June 05, 2023 @11:54AM (#36198164)
Just being able to block spammy Stack Overflow clones from ever appearing in the results is worth the price of admission for me.
Re:Google No Longer Automatically Indexes Websites – WTF? by saltysalt on Monday June 05, 2023 @12:13PM (#36198464)
Here is another, just launched: https://greppr.org/
Re:Google No Longer Automatically Indexes Websites – WTF? by Minor49er on Monday June 05, 2023 @03:34PM (#36202290)
Looks promising, though I noticed that it doesn't encode queries properly when searching. For example, if you go to the homepage and search for "../robots.txt", you'll be redirected to the site's own robots.txt file
Re:Google No Longer Automatically Indexes Websites – WTF? by saltysalt on Monday June 05, 2023 @04:15PM (#36202910)
Thank you kindly for testing, I'll need to fix that one.
Re:Google No Longer Automatically Indexes Websites – WTF? by anderber on Monday June 05, 2023 @07:42PM (#36205966)
Don't they use Google for their results? I'd imagine they'd run into the same issues the article is pointing out.
Re:Google No Longer Automatically Indexes Websites – WTF? by dcow on Tuesday June 06, 2023 @10:45AM (#36213614)
No? At least not like you are implying. Kagi queries multiple data sources and synthesizes results. This means Google’s failure to index does not impact Kagi in the same way as it would DDG (with Bing).
Re:Google No Longer Automatically Indexes Websites – WTF? by winternett on Monday June 05, 2023 @10:26AM (#36196606)
I miss AltaVista so much... It was no frills and only based on page content.
Re:Google No Longer Automatically Indexes Websites – WTF? by BryantD on Monday June 05, 2023 @10:28AM (#36196630)
Those of us who worked there thank you!
Re:Google No Longer Automatically Indexes Websites – WTF? by mech422 on Monday June 05, 2023 @10:34AM (#36196713)
I loved me the 'near' keyword - thanks!
Re:Google No Longer Automatically Indexes Websites – WTF? by masukomi on Monday June 05, 2023 @11:32AM (#36197746)
I both appreciated Alta Vista, and appreciated its office space in Littleton ( i think ) when i worked in it after its passing. ;)
Re:Google No Longer Automatically Indexes Websites – WTF? by lispy4 on Monday June 05, 2023 @12:08PM (#36198410)
Loved everything about Alta Vista, including the logo, and the UI.

I miss 90s Internet in general. It wasn't the ugly battleground and desolation planet that the current net has become.

Re:Google No Longer Automatically Indexes Websites – WTF? by fragmede on Monday June 05, 2023 @12:13PM (#36198469)
Remember Guestbooks? you'd visit a website, and volunteer your name and which country you were from and leave comments. And it wouldn't be a cesspool of spam and porn and XSS attacks? How quaint!
Re:Google No Longer Automatically Indexes Websites – WTF? by lispy4 on Monday June 05, 2023 @12:35PM (#36198861)
Oh gosh, yes! And reading the guestbook was always so fun. An elderly friend of mine passed away in 2018, and in doing a (google) search of him, I found guestbooks he'd signed 20 years ago.
Re:Google No Longer Automatically Indexes Websites – WTF? by zorked on Monday June 05, 2023 @02:29PM (#36200809)
You would look for a thing and the first five pages were random mailing list discussion archives discussing how the thing was 5 years before... Altavista was impressive, but there is a reason why it went away.
Re:Google No Longer Automatically Indexes Websites – WTF? by throwaway894345 on Monday June 05, 2023 @11:03AM (#36197177)
I used DDG for a while, but DDG's quality fell precipitously a few years ago (similar issues where it ignores quotes and won't find pages even if you search for the title string exactly, etc) and I eventually came back to Google which has also been increasingly frustrating.

> I wish there was a search engine that ran like mid 2000s google but with a social media component so you can down vote SEO spammer blogs into oblivion.

There's no way this won't get abused, but the SEO stuff is out of control. Not even spammer blogs, but if you have a quick question like "how do I check tire pressure" you will only get articles that start with a treatise on the entire history of car tires and the answer is deeply buried somewhere in the article. My guess is that Google sees that we're on the page for a longer time than we would spend on pages that just return the answer, and they assume that "more time on page" == "better content" or something.

Re:Google No Longer Automatically Indexes Websites – WTF? by red_admiral on Monday June 05, 2023 @11:13AM (#36197349)
The tire pressure query is exactly the kind of thing that AI should be able to handle easily, though. At which point google has an incentive to sort their competitiveness out.
Re:Google No Longer Automatically Indexes Websites – WTF? by AshamedCaptain on Monday June 05, 2023 @11:52AM (#36198124)
DDG has become ridiculous. They seem to be merging "local", geoIP based results no matter what country I select on the region list (or I disable it). Very often completely unrelated stuff (but local) appears on the 5th or 6th result, midway the first page.

Most egregiously I will search for something very rare (e.g. about programming) and DDG will return me results regarding my city's tourist/visitor info. It's as if it just keeps ignoring words from the search prompt that return no results until it runs out of keywords then it's just the geoIP results.

Re:Google No Longer Automatically Indexes Websites – WTF? by account42 on Tuesday June 06, 2023 @07:09AM (#36210925)
I hate this forced localization so much and its everywhere. The internet used to be a place where you would actually encounter stuff outside your locale.
Re:Google No Longer Automatically Indexes Websites – WTF? by pixodaros on Monday June 05, 2023 @12:17PM (#36198542)
That is because DuckDuckGo started relying almost entirely on Bing for their regular search results after first Yahoo gave up maintaining its own index then Yandex became part of a natio non grata leaving them to choose between partnering with Bing and partnering with Google or creating their own index https://help.duckduckgo.com/duckduckgo-help-pages/results/so...
Re:Google No Longer Automatically Indexes Websites – WTF? by mostlysimilar on Monday June 05, 2023 @11:42AM (#36197937)
> I wish there was a search engine that ran like mid 2000s google but with a social media component so you can down vote SEO spammer blogs into oblivion.

I want this too, but I think an often understated aspect of this issue is that by this point Google has absolutely trashed the web of that era. In these threads people will say "the content you want isn't out there, it's all on social media now" -- and they're largely right, but I think Google is the party most responsible for mutilating the web to the state it is in now, and users fled to social media partly because it seemed like a safe haven.

What we need is a concentrated effort to rebuild the web. Take the best parts of what we've learned and combine with the best parts of what we've left behind and try to build something better, for humans, not for advertisers and hyper-capitalists.

That will take time, energy, and people who remember what we lost and believe we can build something better. A better search engine alone is not enough.

Re:Google No Longer Automatically Indexes Websites – WTF? by JustLurking2022 on Monday June 05, 2023 @12:32PM (#36198798)
The web changed when every kinda slimy business bro realized they could monetize gaming search results. No matter what your fantasy web looks like, be assured, people will game it to the point it's not what you intended.
Re:Google No Longer Automatically Indexes Websites – WTF? by mostlysimilar on Monday June 05, 2023 @12:37PM (#36198902)
If I take that viewpoint on everything I might as well live as a recluse in the woods and avoid people altogether. I have to believe that there are enough of us are out there that genuinely want to build better things for people.
Re:Google No Longer Automatically Indexes Websites – WTF? by noizejoy on Monday June 05, 2023 @05:43PM (#36204267)
The web, just like the real world isn’t static. Becoming and staying intellectually, emotionally and physically mobile may be the only long term strategy to avoid ending up in one or the other dystopia, sooner or later.

When rates of change were slower, you might only have to “move” once in your life, but with increasing rates of change in our human experience, staying nimble is arguably of ever increasing importance.

Re:Google No Longer Automatically Indexes Websites – WTF? by joeythedolphin on Wednesday June 07, 2023 @01:05AM (#36223062)
Yes. This is like water -- keep it moving, find fresh streams.
Re:Google No Longer Automatically Indexes Websites – WTF? by JustLurking2022 on Monday June 05, 2023 @06:41PM (#36205150)
And my point is that there are probably a lot of those motivated people working on the problem today. You make it out as though we've arrived at this state by either lack of effort or competence by Google/Microsoft. My guess is that every time they change the algorithm, the spammers adapt too. That's inevitable and would be just as much of a challenge for your supposed utopia. If you have some secret they don't, there's certainly plenty of money to be made.
Re:Google No Longer Automatically Indexes Websites – WTF? by yomlica8 on Monday June 05, 2023 @12:46PM (#36199076)
It seems like google hardly returns results from traditional forums or blogs which has probably accelerated their decline artificially.
Re:Google No Longer Automatically Indexes Websites – WTF? by Falkon1313 on Monday June 05, 2023 @08:44PM (#36206582)
Largely right, but actually a lot of that stuff is still out there. The personal and hobby pages, forums, blogs, etc.

Google just doesn't know that they exist anymore, or rather doesn't want us to know, because those sites are not commercial enough or big enough.

Almost without fail, no matter what you search for, it tries its best to turn it into a search for a product or service. And those content oriented websites don't fit that, so it just pretends they don't exist.

Re:Google No Longer Automatically Indexes Websites – WTF? by fallingknife on Monday June 05, 2023 @11:48AM (#36198045)
What I want is a "serious mode" that makes it favor primary sources, peer reviewed papers, and raw data. When I search for economic data, I don't want a million news articles referencing pieces of it. I want the raw data release. When I search for some video going viral, I don't want a million videos of journalists talking and showing clips. I want the full raw video.
Re:Google No Longer Automatically Indexes Websites – WTF? by melagonster on Tuesday June 06, 2023 @05:55AM (#36210431)
Google scholar?
Re:Google No Longer Automatically Indexes Websites – WTF? by joeythedolphin on Wednesday June 07, 2023 @12:59AM (#36223022)
Beautifully said! As a thinker of philosophy, I have come to understand that our clip-society is based by design. People can express power over others if they tell you a construction and then show a clip to support it. They really don't want you to see the source/what it is/the truth. They want you want you see what they show you. This problem is accelerating in western societies and it is a fundamental problem of human nature. Journalism is the healthy expression and what we see in today's media is the sickly end.
Re:Google No Longer Automatically Indexes Websites – WTF? by marginalia_nu on Monday June 05, 2023 @01:15PM (#36199543)
> I wish there was a search engine that ran like mid 2000s google but with a social media component so you can down vote SEO spammer blogs into oblivion.

This is sort of what I've been trying to do with Marginalia Search, except I don't really believe a voting system would work. It's far too easy to manipulate. Been playing with the thought of having something like an adblock-list style system where domain shitlists can be collaborated on and shared without being authoritative for the entire search engine.

My search engine is still pretty rough around the edges and limited but I think it works well enough to demonstrate the idea has some merit.

Re:Google No Longer Automatically Indexes Websites – WTF? by Tommstein on Monday June 05, 2023 @06:07PM (#36204665)
> Been playing with the thought of having something like an adblock-list style system where domain shitlists can be collaborated on and shared without being authoritative for the entire search engine.

Even just personal shitlists would be golden and make just about everyone happy.

Re:Google No Longer Automatically Indexes Websites – WTF? by freitzkriesler2 on Monday June 05, 2023 @08:54PM (#36206657)
Will check it out, thanks bud.
Re:Google No Longer Automatically Indexes Websites – WTF? by hnick on Monday June 05, 2023 @09:49PM (#36207052)
Something I've wanted (which probably exists as an extension in Chrome?) for Google searches is a simple blacklist. Just a little button and confirmation next to a result, telling it to never show this blog-spam-ad-laden-SEO-mess of a page to me ever again. Maybe it's an uphill battle, but for some niche topics (like certain games) there are some sites I keep having to scroll past and sometimes accidentally click that are written in SEO-speak and say a lot without saying anything at all.
Re:Google No Longer Automatically Indexes Websites – WTF? by eadler on Tuesday June 06, 2023 @11:54PM (#36222636)
Re:Google No Longer Automatically Indexes Websites – WTF? by flyinghamster on Monday June 05, 2023 @11:12AM (#36197342)
I'm noticing this with DDG as well. :( I guess the powers that be have decided that information must be hidden.
Re:Google No Longer Automatically Indexes Websites – WTF? by pixodaros on Monday June 05, 2023 @12:14PM (#36198484)
Its even stupider than that. There are only two major, publicly available web indexes in the USA, Google's and Bing's. After 24 February 2022, DuckDuckGo ended their partnership with Yandex, and since then they say "we have more traditional links and images in our search results too, which we largely source from Bing" https://help.duckduckgo.com/duckduckgo-help-pages/results/so...
Re:Google No Longer Automatically Indexes Websites – WTF? by HWR_14 on Monday June 05, 2023 @12:34PM (#36198845)
The web indexes from Google and Bing are available publicly? I can pull it down from somewhere and try to make a search engine?
Re:Google No Longer Automatically Indexes Websites – WTF? by pixodaros on Monday June 05, 2023 @01:01PM (#36199342)
Bing at least license their indexes to partners on a commercial basis, as did Yahoo until they gave up indexing the web. I am sure that the NSA, the Chinese government, the ahrefs website, and other organizations have comprehensive indexes of the web which they don't share in this way.

Mojeek seems to be the independent, non-paywalled search engine with the biggest index, for an overview see https://seirdy.one/posts/2021/03/10/search-engines-with-own-...

Re:Google No Longer Automatically Indexes Websites – WTF? by Der_Einzige on Monday June 05, 2023 @11:26AM (#36197615)
Be careful! The Google search guys will come on HN and gaslight you about this, claiming that the advanced search functionality works perfectly and it's simply user error.

We know it's not, but expect them to try to tell you you're imagining things.

Re:Google No Longer Automatically Indexes Websites – WTF? by ransackdev on Monday June 05, 2023 @11:48AM (#36198038)
Funny that people call these "bugs" as if anything related to google search happens on accident.

They don't need to waste the eng resources or infrastructure on rock solid search anymore, they own the market and got all the users into their funnel of products, most locked in for life.

Search results still show sponsored listings, they still have all the users, and all the profit, and a lot less of the profit sucking operational costs it took to be good at what made them a household name, search

Re:Google No Longer Automatically Indexes Websites – WTF? by passion__desire on Monday June 05, 2023 @01:12PM (#36199517)
Is google trading "accuracy" for computation cost at the same time inserting junk results into the results?
Re:Google No Longer Automatically Indexes Websites – WTF? by ransackdev on Saturday June 10, 2023 @09:12PM (#36276978)
They aren’t inserting junk, they just don’t do anything to rank quality results above the junk anymore. The junk was always there, on page 2 and beyond, and who would ever need to hit those pages. Nowadays I’ll be 15 pages deep and so far off base of the search term that I could write a better index with curl and regex.

The issue isn’t that they are watering down the cream in the milk, it’s that the cream isn’t part of the milk at all now.

Re:Google No Longer Automatically Indexes Websites – WTF? by wolpoli on Monday June 05, 2023 @10:11PM (#36207185)
This is a good point. Google got to the top by having the best search result back when it mattered. It now no longer matters.
Re:Google No Longer Automatically Indexes Websites – WTF? by 13of40 on Monday June 05, 2023 @12:52PM (#36199177)
I think google does OK with the syntax it still supports for text queries, but if you switch to the images tab it just thows all of that stuff out the window. I would love to be able to search for "cat eating watermelon" or whatever and only get results with cats eating watermelon, ordered by the proximity of that text to the image returned. Hopefully AI is going to do something for that, but the state of the art, as embodied by the biggest player (Google) is shamefully deficient.
Re:Google No Longer Automatically Indexes Websites – WTF? by bbarnett on Monday June 05, 2023 @01:04PM (#36199383)
Use verbatim mode, under tools, after your initial search. They broke it, but it still helps.
Re:Google No Longer Automatically Indexes Websites – WTF? by akvadrako on Monday June 05, 2023 @04:45PM (#36203341)
Those operators are no longer supported. You can use Verbatim mode, which is more like the old behavior.
Re:Google No Longer Automatically Indexes Websites – WTF? by burnte on Thursday June 08, 2023 @11:33AM (#36243623)
What? Are you serious?