News for hackers. Stuff that matters.
Re:Google is ending full-web search for niche search engines by throwaway_20357 on Friday January 23, 2026 @04:52AM (#46730516)
What are some of the niche search engines build on Google's index affected by this?
Re:Google is ending full-web search for niche search engines by doublerabbit on Friday January 23, 2026 @05:01AM (#46730581)
Kagi
Re:Google is ending full-web search for niche search engines by nemosaltat on Friday January 23, 2026 @05:46AM (#46730928)
> Kagi This seems to be true, but more indirectly. From Kagi’s blog [0] which is a follow up to a Kagi blog post from last year [1].

[0]> Google: Google does not offer a public search API. The only available path is an ad-syndication bundle with no changes to result presentation - the model Startpage uses. Ad syndication is a non-starter for Kagi’s ad-free subscription model.[^1]

[0]> The current interim approach (current as of Jan 21, 2026)

[0]> Because direct licensing isn’t available to us on compatible terms, we - like many others - use third-party API providers for SERP-style results (SERP meaning search engine results page). These providers serve major enterprises (according to their websites) including Nvidia, Adobe, Samsung, Stanford, DeepMind, Uber, and the United Nations.

I’m an avid Kagi user, and it seems like Kagi and some other notable interested parties have _already_ been unable to do get what they want/need with Google’s index.

[0]> The fact that we - and companies like Stanford, Nvidia, Adobe, and the United Nations - have had to rely on third-party vendors is a symptom of the closed ecosystem, not a preference.

Hopefully someone here can clarify for me, or enumerate some of these “third-party vendors” who seem like they will/might/could be directly affected by this.

[0] antibabelic > relevant https://blog.kagi.com/waiting-dawn-search [1] https://blog.kagi.com/dawn-new-era-search > [^1]: A note on Google’s existing APIs: Google offers PSE, designed for adding search boxes to websites. It can return web results, but with reduced scope and terms tailored for that narrow use case. More recently, Google offers Grounding with Google Search through Vertex AI, intended for grounding LLM responses. Neither is general-purpose index access. Programmable Search Engine is not designed for building competitive search. Grounding with Google Search is priced at $35 per 1,000 requests - economically unviable for search at scale, and structured as an AI add-on rather than standalone index syndication. These are not the FRAND terms the market needs

Re:Google is ending full-web search for niche search engines by tpetry on Friday January 23, 2026 @06:24AM (#46731200)
I believe they try to indirectly say they are using SerpApi or a similar product that scrapes Google search results to use them. And other big ones use it too so it must be ok...

That must be the reason why they limit the searches you can do in the starter plan. Every SerpApi call costs money.

Re:Google is ending full-web search for niche search engines by sixhobbits on Friday January 23, 2026 @07:35AM (#46731741)
Google is also suing SerpAPI

And I can't prove correlation but they refused to index one of my domains and I think it _might_ be because we had some content on there about how to use SerpAPI

Re:Google is ending full-web search for niche search engines by monooso on Friday January 23, 2026 @07:00AM (#46731478)
Kagi does not use Google's search index. From their post which made the front page of HN yesterday [1]:

> Google does not offer a public search API. The only available path is an ad-syndication bundle with no changes to result presentation - the model Startpage uses. Ad syndication is a non-starter for Kagi’s ad-free subscription model.

[1]: https://news.ycombinator.com/item?id=46708678

Re:Google is ending full-web search for niche search engines by jsnell on Friday January 23, 2026 @07:25AM (#46731671)
They then go on to say that they pay a 3rd party company to scrape Google results (and serve those scraped results to their users). So their search engine is indeed based on unauthorized and uncompensated use of Google's index.

But since they're not using/paying for a supported API but just taking what they want, they indeed are unlikely to be impacted by this API turndown.

Re:Google is ending full-web search for niche search engines by DangitBobby on Friday January 23, 2026 @09:30AM (#46732904)
Congrats on saying that in the most one-sided way possible. Google makes it literally impossible for them to pay for access to search results to make the product they want (customizable subscription search with no ads), and Google also is the de-facto globally sanctioned crawler because they are the only search engine anyone gives a shit about, and also sites need to be indexed by them to survive. In short, Google owns the river and sells the boats, and the public built a wall around it. Google is in a monopoly position in search.
Re:Google is ending full-web search for niche search engines by nova22033 on Friday January 23, 2026 @10:10AM (#46733414)
>In short, Google owns the river and sells the boats, and the public built a wall around it.

That would be a monopoly if there was only 1 river in the whole world.

Re:Google is ending full-web search for niche search engines by DangitBobby on Friday January 23, 2026 @01:51PM (#46736215)
Yeah I mean think whatever you need to for the metaphor to work.
Re:Google is ending full-web search for niche search engines by Ferret7446 on Friday January 23, 2026 @05:19PM (#46738749)
They have a monopoly on their own search results. There's nothing stopping anyone from making their own (hell, a poster did so in the comments above). God forbid we aren't entitled access to the fruits of their labor; the reason you want it isn't because you can't make it (again, see above). It's because making it good is hard, and you want the good results without yourself putting in the effort to make it
Re:Google is ending full-web search for niche search engines by ipaddr on Friday January 23, 2026 @12:45PM (#46735368)
They get results from another provider who has authorized access. Google doesn't provide search results to unauthorized requests as many on tor have experienced.
Re:Google is ending full-web search for niche search engines by direwolf20 on Friday January 23, 2026 @05:01PM (#46738564)
Residential proxies are also cheaper than you might realize.
Re:Google is ending full-web search for niche search engines by jsnell on Saturday January 24, 2026 @11:51AM (#46745127)
No. They pay SerpApi to scrape Google. And SerpApi is currently being sued by Google for unauthorized scraping.

Kagi did make comments for years implying that they had a deal with Google for search results, but their latest blog post makes it clear that is not true and was never true.

Re:Google is ending full-web search for niche search engines by marginalia_nu on Friday January 23, 2026 @05:06AM (#46730620)
They published this the other day:

https://blog.kagi.com/waiting-dawn-search

Which saw some discussion on HN.

Re:Google is ending full-web search for niche search engines by embedding-shape on Friday January 23, 2026 @05:30AM (#46730809)
> some discussion

~450 score, ~247 comments and still on /best ("Most-upvoted stories of the last 48 hours"):

https://news.ycombinator.com/item?id=46708678 - "Waiting for dawn in search: Search index, Google rulings and impact on Kagi"

Re:Google is ending full-web search for niche search engines by pell on Friday January 23, 2026 @05:07AM (#46730626)
I think Kagi buys search engine results from SERP vendors who typically scrape Google’s results and offer an API experience on top of it.
Re:Google is ending full-web search for niche search engines by echelon on Friday January 23, 2026 @05:18AM (#46730714)
No wonder Kagi is angry.

Google is a monopoly across several broad categories. They're also a taxation enterprise.

Google Search took over as the URL bar for 91% of all web users across all devices.

Since this intercepts trademarks and brand names, Google gets to tax all businesses unfairly.

Tell your legislators in the US and the EU that Google shouldn't be able to sell ads against registered trademarks (+/- some edit distance). They re-engineered the web to be a taxation system for all businesses across all categories.

Searching for Claude -> Ads in first place

Searching for ChatGPT -> Ads in first place

Searching for iPhone -> Ads in first place

This is inexcusable.

Only searches for "ChatGPT versus", "iPhone reviews", or "Nintendo game comparison" should allow ads. And one could argue that the "URL Bar" shouldn't auto suggest these either when a trademark is in the URL bar.

If Google won't play fair, we have to kill 50% of their search revenue for being egregiously evil.

If you own a trademark, Google shouldn't be able to sell ads against you.

--

Google's really bad. Ideally we'd get an antitrust breakup. They're worse than Ma Bell. I wouldn't even split Google into multiple companies by division - I'd force them to be multiple copies of the same exact entity that then have to compete with each other:

Bell Systems -> {BellSouth, Bell Atlantic, Southwestern Bell, ...}

Google -> {GoogleA, GoogleB, GoogleC, ...}

They'd each have cloud, search, browser, and YouTube. But new brand names for new parent companies. That would create all-out war and lead to incredible consumer wins.

Re:Google is ending full-web search for niche search engines by marginalia_nu on Friday January 23, 2026 @05:40AM (#46730894)
Could probably argued that search access is an essential facility[1], though it doesn't appear antitrust law has anywhere near the same sort of enforcement it did in the past.

[1] https://en.wikipedia.org/wiki/Essential_facilities_doctrine

Re:Google is ending full-web search for niche search engines by throwaway290 on Friday January 23, 2026 @07:29AM (#46731683)
what stops Kagi from indexing internet and makes them pay some guys to scrape search results from Google? one guy at Marginalia can do it and entire dev team at a PAID search engine can't?
Re:Google is ending full-web search for niche search engines by DangitBobby on Friday January 23, 2026 @09:36AM (#46732983)
As we've seen here on HN on the AI boom, it's not wonderful when a bunch of companies all use bots to scrape the entire web. Many sites only allow Google scrapers in robots.txt and the public will fight you hard if you scrape them without permission. It's just one of those things where it would be better for everyone if search engines could pay for access to the work that's done only once.
Re:Google is ending full-web search for niche search engines by echelon on Friday January 23, 2026 @09:52AM (#46733169)
> Many sites only allow Google scrapers in robots.txt and the public will fight you hard if you scrape them without permission.

This just lets a monopoly replace the website instead of distributing power and fostering open source. The same monopoly that was already bleeding off the web's utility and taxing it.

Re:Google is ending full-web search for niche search engines by mrweasel on Friday January 23, 2026 @10:26AM (#46733623)
I don't know about others, but we have special rules for Google, Bing, and a few others, rate-limiting them less than some random bot.

The problem is scrapers (mostly AI scrapers from what we can tell). They will pound a site into the ground and not care and they are becoming increasingly good at hiding their tracks. The only reasonable way to deal with them is to rate-limit every IP by default and then lifting some of those restrictions on known, well behaving bots. Now we will lift those restrictions if asked, and frequently look at statistics to lift the restrictions from search engines we might have missed, but it's an up hill battle if you're new and unknown.

Re:Google is ending full-web search for niche search engines by thewebguyd on Friday January 23, 2026 @11:20AM (#46734314)
> If you own a trademark, Google shouldn't be able to sell ads against you.

This is frustrating even from a consumer perspective. Before I ran adblock everywhere, I couldn't stand that typing in a specific company I was looking for would just serve ads from any number of related brands that I wasn't looking for that were competitors.