News for hackers. Stuff that matters.
Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @07:18AM (#40510691)
As a user of Kagi and search.marginalia.nu I can tell you:

Quite a bit.

So much that now that I have what "everyone" asked Google for for years - that is blacklists - I hardly use them.

Why? Because with Kagi I get much better results out of the box.

I am fairly sure Googlers will tell me there are multiple safeguards to prevent the inclusion of Google ads from affecting ranking, to which I just have to say that the results speak for themselves.

Please note: I have only used Kagi for two years. I am only one user. But I am a user with 20 years of experience with Google and that got to count for something.

Re:Google Search document leak reveals inner workings of ran... by esperent on Wednesday May 29, 2024 @08:16AM (#40511121)
Kagi is worth the money, but it isn't magic. It's about as good as Google was ~five years ago, before they made all the search operators stop work. There's also a whole bunch of things it's worse at that Google - especially local search and shopping. Plus I still get plenty of blogspam and AI generated crap from Kagi.
Re:Google Search document leak reveals inner workings of ran... by mozman on Wednesday May 29, 2024 @08:30AM (#40511230)
The search operators makes big difference in result quality, I also don’t like how Google now returns zero results for something obscure. In the past I could find something peripheral and eventually get to what I was looking for.
Re:Google Search document leak reveals inner workings of ran... by orangevelcro on Wednesday May 29, 2024 @10:22AM (#40512318)
Exactly! It's so frustrating because the results were usually useful enough to at least point me in the right direction or help me figure out what I was doing wrong to find what I wanted.
Re:Google Search document leak reveals inner workings of ran... by Nevolihs on Wednesday May 29, 2024 @07:55AM (#40510947)
I actually use pinning, blocking and raising/lowering the value of individual sites every day. I wish this is the direction search engines went in the first place and it's the direction I hope Kagi continues. I want a personalized search engine that's personalized by me, not by a company trying to profile me and make money off of my clicks.
Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @09:15AM (#40511640)
I am all for Kagi keeping that feature. If for nothing else then to rub it in the face of every googler who have argued that it was impossible.

And if you use it I am happy, that gives Kagi an incentive to keep it around.

I'm just saying that for me the results are so good out of the box that with a couple of exceptions I never had to block anything.

Re:Google Search document leak reveals inner workings of ran... by the_snooze on Wednesday May 29, 2024 @09:42AM (#40511920)
When each user can personalize the results themselves, you make SEO completely impractical because they can no longer target a single monolithic algorithm controled by one entity. Websites would actually have to have organic appeal to users, who get the final say to hide away bad sites from the results page (looking at you, Quora, Pinterest, and Fandom).
Re:Google Search document leak reveals inner workings of ran... by beeboobaa3 on Wednesday May 29, 2024 @07:59AM (#40510990)
Is kagi good for finding things like old forum posts (not reddit)? I know some of those sites are still up but google seems to ignore them.
Re:Google Search document leak reveals inner workings of ran... by stuffoverflow on Wednesday May 29, 2024 @08:23AM (#40511175)
In my experience kagi is decent, definitely gives more forum results than google. I've found yandex to be the best at finding all kinds of forum /discussion sites.
Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @09:20AM (#40511696)
There is a seperate "lens" (think like "images", "videos" and "news" in Google, only there are more and you can create your own) for "Small Web" which only includes what they describe as "Results that favor noncommercial domains and topics".

(Other standard lenses include

- Forums

- Fediverse forums

- Usenet/archive

and I think 7 others.)

Re:Google Search document leak reveals inner workings of ran... by nalinidash on Wednesday May 29, 2024 @09:29AM (#40511794)
Try search.marginalia.nu

From the website about: "This is an independent DIY search engine that focuses on non-commercial content, and attempts to show you sites you perhaps weren't aware of in favor of the sort of sites you probably already knew existed."

Re:Google Search document leak reveals inner workings of ran... by amelius on Wednesday May 29, 2024 @08:08AM (#40511054)
How do you know that Kagi won't become as bad as Google at some point?
Re:Google Search document leak reveals inner workings of ran... by spacebanana7 on Wednesday May 29, 2024 @08:22AM (#40511164)
Also it’s unlikely Kagi will ever become big enough for SEO people to specifically target with manipulative content.

Even if they got 100 million active paying users it’d still be a tiny fraction of overall search traffic.

Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @09:34AM (#40511833)
If anything, Kagi has proven that Googles "SEO ruined it" argument isn't valid.

In the beginning, Kagi was mostly just API calls to Google and Bing AFAIK. Their results were still better for some reason, probably because they didn't have to consider how valuable the ads on garbage site were before throwing the site out of my results.

Re:Google Search document leak reveals inner workings of ran... by spacebanana7 on Wednesday May 29, 2024 @10:04AM (#40512154)
I agree with you to some extent. For example, there's no reason I can think of why Google couldn't allow user selected domain blacklisting or custom bangs.

But I fear that if a large search engine copied some of Kagi's filters/lenses for non-commercial blogs & forums then SEO agencies would dedicate immense resources to polluting them.

Re:Google Search document leak reveals inner workings of ran... by freediver on Wednesday May 29, 2024 @11:57PM (#40519930)
Not possible by definition. Kagi Small Web is non commercial sites only, by definition. And the list is fixed (and open source). There is nothing SEO can do to infiltrate it.
Re:Google Search document leak reveals inner workings of ran... by Kye on Wednesday May 29, 2024 @10:37AM (#40512441)
It doesn't have to be big if it's rich.

People will target Kagi because it's packed to the brim with people who have $5-10/month to spend on a search engine, and likely have $5-10/month to spend on other things too.

Re:Google Search document leak reveals inner workings of ran... by p3rls on Wednesday May 29, 2024 @09:27AM (#40511769)
Why would they? The same manipulative techniques that work in my niche on Google seem to work even better on Kagi
Re:Google Search document leak reveals inner workings of ran... by breakfastduck on Wednesday May 29, 2024 @08:13AM (#40511088)
We don't, but a model where a user pays for a service rather than being free and ad supported is significantly less likely to enact user unfriendly changes.

If the way you make money is by convincing people to pay, you are highly incentivised to make the product good, especially where there are many other free competitors who are ad supported.

Re:Google Search document leak reveals inner workings of ran... by glitchc on Wednesday May 29, 2024 @08:21AM (#40511156)
Tell that to cable companies, sports broadcasts and now streaming services. The one incontrovertible fact of modern society is that ads eat everything else. No other revenue source compares.
Re:Google Search document leak reveals inner workings of ran... by karma_pharmer on Wednesday May 29, 2024 @08:33AM (#40511260)
ads eat everything else

Well duh. Because it's the only value you're allowed to collect from retail customers without having to deal with either chargebacks or kyc.

We've deliberately made every other form of micropayment infeasible. It shouldn't be surprising that the only one left is insanely popular.

Re:Google Search document leak reveals inner workings of ran... by sseagull on Wednesday May 29, 2024 @09:28AM (#40511783)
Partly. But also because it's just a way to make more money. If you make X from payments, and you can make Y from ads, then X+Y > X.

You might lose some people, but probably not many - we've been conditioned to really accept ads everywhere.

Re:Google Search document leak reveals inner workings of ran... by glitchc on Wednesday May 29, 2024 @10:43AM (#40512506)
Also because Y>>X.
Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @09:31AM (#40511806)
Kagi is worth the price every month, unlike a number of other things I have supported it is not an investment for me in the hope that it will one day be worth it but rather a service that I pay a small sum for which in turns removes lots of frustration from my life every month.

If they do become bad then at least I have had a fantastic search engine for another few years of my life like I had from 2002 until 2009 ish.

And also, already at this point, they and marginalia has proven that it isn't impossible to enter the search engine marked even now. This was long considered impossible, at least here on HN.

Re:Google Search document leak reveals inner workings of ran... by duckmysick on Wednesday May 29, 2024 @12:21PM (#40513683)
What's the argument you're trying to make?

That because there's a chance Kagi will become bad, there's no point in using it now and thus we should stick to Google, which is already bad? That doesn't make sense.

The same line of thought can be applied to anything. We don't know what will happen in the future, therefore we can't be sure that things won't go bad. Is there even a way to have such a guarantee?

Assuming I don't want to use Google (because it's bad) and I won't use Kagi or Perplexity or others (because they might become bad), what's the realistic solution? Roll my own search engine? I don't have resources and I don't trust the future me to maintain it.

Re:Google Search document leak reveals inner workings of ran... by abhijat on Wednesday May 29, 2024 @07:34AM (#40510793)
I switched to Kagi in June last year. I just realized I tried it initially because I wanted to try out blocking sites in search results, and I have only ever needed to block three domains.
Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @07:41AM (#40510843)
Thats exactly what I am talking about.

Kagi is kind of like Google in 2009, seriously good coverage, good ranking

... but also:

- more modern

- more features (summarizer, bangs like in DDG, FastGPT and probably a few I forgot)

- blocklists for websites (and also options to pin, raise and lower)

- with actual support: report a bug and you get an answer from a real engineer, a follow up when it is fixed and a shout out in the relevant release notes

- no tracking

Re:Google Search document leak reveals inner workings of ran... by dustincoates on Wednesday May 29, 2024 @08:13AM (#40511100)
I like Kagi a lot (just look at my comment history), but I'm letting my subscription lapse when it comes time to renew. I've found myself going to Google a lot more often, and I'm finding more and more transparently spammy sites in the Kagin index. Some, for example, are clearly Gen AI created.

If I were a rich man, I would probably keep my subscription just to support a Google competitor. Alas, I'm not, and so I'll be going back to Google.

Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @09:23AM (#40511721)
I see your point. I was not always this well paid.

Did you try blocking the problematic sites and if it didn't work, did you file support requests?

Re:Google Search document leak reveals inner workings of ran... by dustincoates on Wednesday May 29, 2024 @10:20AM (#40512302)
I didn't block them, because they were rarely the same site repeatedly, so I'm not sure it would help.

I didn't on support requests, either, perhaps I should. I have before, and the team did a great job at addressing what wasn't working.

Re:Google Search document leak reveals inner workings of ran... by freediver on Wednesday May 29, 2024 @11:51PM (#40519899)
Have you reported any of this to Kagi? (things that are reported usually get fixed)
Re:Google Search document leak reveals inner workings of ran... by nolist_policy on Wednesday May 29, 2024 @07:57AM (#40510973)
Is it only me or do these constant Kagi ads on HN sound fishy?
Re:Google Search document leak reveals inner workings of ran... by danielheath on Wednesday May 29, 2024 @08:30AM (#40511239)
It’s not surprising that folks who pay for a service when there’s a free alternative are pretty serious fans.
Re:Google Search document leak reveals inner workings of ran... by ysavir on Wednesday May 29, 2024 @08:02AM (#40511003)
I wouldn't necessarily call them fishy, but I am very tired of them. They have a very evangelizing tone. But I think they're ultimately just people excited about the tool they're using and wanting to share it with others.
Re:Google Search document leak reveals inner workings of ran... by tomrod on Wednesday May 29, 2024 @08:18AM (#40511137)
I took a screenshot years ago where 10/14 of the viewable top headlines on my screen where positive Google discussion. From an advertising perspective it was all earned marketing (satisfied customers speaking highly).

While these situations could be a pg-style astroturf submarine, or they could be satisfied customers (the best kind of advertising), I wouldn't necessarily say fishy (you can look at the satisfied users' previous contributions to make that judgment yourself! :)).

Personally, I've not used Kagi, but I hear positive things from people I trust that use it. So I'll likely try it in the future.

Re:Google Search document leak reveals inner workings of ran... by drpossum on Wednesday May 29, 2024 @08:03AM (#40511014)
Maybe they're not ads but people who genuinely like the service?
Re:Google Search document leak reveals inner workings of ran... by epr on Wednesday May 29, 2024 @08:12AM (#40511082)
Did we not all evangelize Google in it's early days?

Also, none of these accounts saying nice things appear to be bots or kagi-focused in any way, so I think it's safe to assume they do actually just like it.

Re:Google Search document leak reveals inner workings of ran... by orangevelcro on Wednesday May 29, 2024 @10:26AM (#40512351)
I don't know...my spidey sense has been going off a bit.

Kagi has a free trial, but you have to pay, which is the difference between it and early Google.

Of course, now we have Google ads instead, so who knows, maybe not bots.

Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @06:13PM (#40517681)
Go check my history. Send me a mail and I'll send a photo of the fields and the wind breaks here that you can geolocate.

I am definitely not a bot.

I am however extremely fed up with Google. And equally thankful that I have found something that works as well as old Google (or better).

Re:Google Search document leak reveals inner workings of ran... by breakfastduck on Wednesday May 29, 2024 @08:14AM (#40511104)
Google Search being a bit rubbish has been in the zeitgeist for a while, it's not surprising that people then talk about an alternative they've found that is much better in their experience
Re:Google Search document leak reveals inner workings of ran... by mannycalavera42 on Wednesday May 29, 2024 @08:10AM (#40511066)
kagi is the new crossfit
Re:Google Search document leak reveals inner workings of ran... by JTyQZSnP3cQGa8B on Wednesday May 29, 2024 @10:22AM (#40512313)
I never say it but here it is: for the price of 2 packs of cookies, I went from being a 1x programmer to a 1.5x programmer without doing anything. If the results are good, it’s good for me and my job which brings me way more money and satisfaction than $10.

The alternative is Searx and I may try it sometimes, but so far Kagi is cheap and very efficient for me (C++ coding and other languages).

Re:Google Search document leak reveals inner workings of ran... by elorant on Wednesday May 29, 2024 @08:26AM (#40511206)
I use FastGPT quite often although I’m not a subscriber to Kagi itself. For me it’s everything an AI search engine should be. Here is your answer, and here are a bunch of links to research further. Something that works without making the web obsolete. Not like the walled off garden of OpenAi which often hallucinates links, or Google’s “I through everything at the wall to find what sticks” effort.
Re:Google Search document leak reveals inner workings of ran... by karma_pharmer on Wednesday May 29, 2024 @08:27AM (#40511209)
Kagi is simply reselling google search results.
Re:Google Search document leak reveals inner workings of ran... by ptman on Wednesday May 29, 2024 @08:38AM (#40511305)
Re:Google Search document leak reveals inner workings of ran... by karma_pharmer on Wednesday May 29, 2024 @09:33AM (#40511822)
... and 99% of the results come from just one of those.
Re:Google Search document leak reveals inner workings of ran... by super256 on Wednesday May 29, 2024 @08:36AM (#40511288)
Re:Google Search document leak reveals inner workings of ran... by karma_pharmer on Wednesday May 29, 2024 @09:34AM (#40511838)
Then why does kagi have the same artificially-massive github boost that google does and no other engine (bing, duckduckgo) does?

They're just reselling google searches. With maybe 1% salted in from elsewhere for deniability.

Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @09:37AM (#40511862)
Because that is what most of us prefer?

Or at least I thought so?

But with Kagi you can always lower github results if you want to. Or boost other sites that you prefer.

Re:Google Search document leak reveals inner workings of ran... by karma_pharmer on Wednesday May 29, 2024 @11:47PM (#40519883)
> Because that is what most of us prefer?

If that is true why hasn't Bing noticed it?

The github bias in google results is huge compared to bing. Just try running the same search on both sites. I find it hard to believe that the Bing engineers simply haven't noticed this.

Re:Google Search document leak reveals inner workings of ran... by scutrell on Wednesday May 29, 2024 @08:48AM (#40511388)
I was excited to try Kagi, but I couldn't justify the cost. I find DDG with the occasional Google search to function almost as well. I'll try Kagi again at some point, but it wasn't the panacea people here made it out to be
Re:Google Search document leak reveals inner workings of ran... by packetlost on Wednesday May 29, 2024 @09:20AM (#40511684)
I dunno, the first thing I did was blacklist G**ksf*rG**ks from my search results (and others, of course) and I couldn't be happier.
Re:Google Search document leak reveals inner workings of ran... by p3rls on Wednesday May 29, 2024 @09:26AM (#40511746)
Kagi is the same garbage as google in my niche. Even worse, maybe. It looks like it's weighing backlinks and SEO garbage even higher. Well done.

I don't know how people keep talking about it. The results, as you say, speak for themselves.

Re:Google Search document leak reveals inner workings of ran... by eitland on Wednesday May 29, 2024 @09:35AM (#40511843)
Well, don't use it then.

I am happy for alternatives, otherwise I guess Kagi wouldn't improve so fast in areas I care about.

Re:Google Search document leak reveals inner workings of ran... by p3rls on Wednesday May 29, 2024 @09:59AM (#40512102)
I won't use it. I am pushing back on your belief that it has figured something out about anything, because to me it looks like Kagi takes Google's results and then makes them worse. I have total shit ranking #1 on Kagi and Google. On other pages I am struggling to break into the top 10 where I should be right under wikipedia in a just world.

I think many people whose lives don't revolve around these things (as in your finances/mortgage are not dependent on google search) get weird magical views about search engines where it's sorta like a modern delphic oracle where the magic works as long as you believe in it. Of course I have the opposite problem where Google is angry chthonic god to be supplicated/scorned.

Oh well, I do know I will not be worshipping a new god that cannot even destroy the abominations sitting at the top of the rankings.

Re:Google Search document leak reveals inner workings of ran... by bitcharmer on Wednesday May 29, 2024 @11:06AM (#40512722)
> Kagi takes Google's results and then makes them worse

And this is how I know you haven't used Kagi at all.

Re:Google Search document leak reveals inner workings of ran... by p3rls on Wednesday May 29, 2024 @12:19PM (#40513636)
> And this is how I know you haven't used Kagi at all.

I guess the identical search results in my niche are an example of convergent enshittification of both Kagi and Google then

Re:Google Search document leak reveals inner workings of ran... by bitcharmer on Wednesday May 29, 2024 @02:11PM (#40514958)
Ok, if that's the case then I give up. Totally possible for different people to have wildly different experience.
Re:Google Search document leak reveals inner workings of ran... by oooyay on Wednesday May 29, 2024 @05:29PM (#40517244)
> I won't use it. I am pushing back on your belief that it has figured something out about anything, because to me it looks like Kagi takes Google's results and then makes them worse.

Kagi is pretty clear about how they do what they do: https://help.kagi.com/kagi/search-details/search-sources.htm.... You're half right, except that Google is not a vertical that they leverage for their index. Their solution works for me, but I'm also a techy. Anecdotally, my partner uses it, and she is not a techy.

> Oh well, I do know I will not be worshipping a new god that cannot even destroy the abominations sitting at the top of the rankings.

The only way this statement applies (to me) is if you can optimize for Kagi. The only way I can think of to do that is organically, but admittedly I haven't put much effort into it because the magic of Kagi's flavor of search is that it puts me in near proximity to information density and sources that I already trust. The real proving grounds for Kagi are a decade out.

Re:Google Search document leak reveals inner workings of ran... by p3rls on Wednesday May 29, 2024 @06:38PM (#40517913)
Lol, it's just complete bullshit marketing on that page. But I do find it fascinating how they obfuscate what is actually happening behind the scenes and cannot come out and clearly say how their technology works, or that they are using google's index.

>Kagi is known for delivering a unique flavor of high-quality search results, sourced from our own web index (internally named "Teclis") and news index (internally named "TinyGem").

The marketing at least is pretty good -- I haven't seen this many committed devotees since the iPhone was announced.

But here on the ground, I can type a few queries and reveal that Kagi is the same shit as Google. Perhaps you are seeing some manually adjusted queries?

P.S. there are lots of other concerns about Kagi too, I'm just talking about the search https://hackers.town/@lori/112255132348604770

Re:Google Search document leak reveals inner workings of ran... by eitland on Thursday May 30, 2024 @05:22AM (#40521673)
> But here on the ground, I can type a few queries and reveal that Kagi is the same shit as Google.

Here on the ground in Norway I just say "lucky you" if Google serves the same results as Kagi.

My experience is Google expands my search terms beyond recognition and I am absolutely and utterly fed up with starting to read an article about a problem I searched for, getting a result from a trustworthy site, starting to read only to realize Google secretly behind my back expanded my search from "unpopular-ancient-javascript-framework-from-2021" to "currently-popular-js-framework-with-kinda-similar-name".

My time is valuable. Not having to babysit my search engine is something I happily pay for.

Could probably get my employer to pay for it too.

If Google by some miracle has you in an experimental group where they don't mess with your queries, or if you have time to babysit it - more power to you.

Re:Google Search document leak reveals inner workings of ran... by p3rls on Thursday May 30, 2024 @10:30AM (#40524197)
Why do Kagi supporters keep insisting I am supporting Google? I save all my best vituperative comments for how Google has destroyed the internet.

I am only saying Kagi is just as shitty as Google for my queries and certainly is using their index based off the 30 or so searches I made. I am not in a technical niche however, so maybe they're using something different for that or you saw a manually-adjusted query?

But for the regular, everyday people queries, yeah 100% it's just google bro.

Re:Google Search document leak reveals inner workings of ran... by eitland on Friday May 31, 2024 @12:45AM (#40531594)
Ok, thanks for the clarification.

Let me also clarify: I am absolutely not denying that Kagi uses Googles index.

I am just pointing out that for some reason when I search with Google I get lots and lots of results for things I didn't search for. For a while I kept them away by sprinkling doublequotes everywhere but the last few years, no amount of double quotes or even the verbatim operator works anymore.

With Kagi however not only does doublequotes work (for now at least), - I usually don't even need to apply them, my queries work just like they used to do in "old" Google back in 2009.

That is what I pay Kagi for.

I can be totally open that for what I know the reason might be as simple as the fact that Kagis API access gets results directly from Googles search API without passing through what I think of as a "search broadening" function and without applying A/B testing experiments to the results that come back.

I don't care. I get my results.

And if someone come along and present a better alternative that uses only their own independent index, I'll be inclined to pay even more just because I'd love to see it. (I was a marginalia supporter for months and might become again in the future.)