News for hackers. Stuff that matters.
Re:SearXNG is a free internet metasearch engine by sitkack on Friday April 05, 2024 @07:08PM (#39948360)
Run this on your personal internet connection, risk not being able to use a search engine.
Re:SearXNG is a free internet metasearch engine by mostlysimilar on Friday April 05, 2024 @07:12PM (#39948382)
Elaborate?
Re:SearXNG is a free internet metasearch engine by marginalia_nu on Friday April 05, 2024 @07:17PM (#39948416)
Virtually all public search engine endpoints see an insane amount of bot activity, often several queries per second.

If you delegate queries to e.g. google or bing at that rate, you'll be ip blocked in a heartbeat.

Re:SearXNG is a free internet metasearch engine by mostlysimilar on Friday April 05, 2024 @07:21PM (#39948447)
Ah duh, for some reason my mind didn't go to hosting the search instance locally and I misunderstood.

btw thank you for Marginalia! The spirit of the small web is very important to me.

Re:SearXNG is a free internet metasearch engine by RaisingSpear on Saturday April 06, 2024 @12:05AM (#39949849)
Search engines: they scrape the web, but get narky when scraped themselves.
Re:SearXNG is a free internet metasearch engine by marginalia_nu on Saturday April 06, 2024 @04:33AM (#39950956)
Difference is a crawler paces the requests, respects robots.txt and rate limits, and doesn't typically invoke 50-100MB disk I/O per request.

Like I don't mind automated access to my search engine, I even offer a public API to the effect, that you can in fact hook into SearXNG. What I mind is when one jabroni with a botnet decides their search traffic is more important than everyone else's and grabs all the compute for himself via a sybil attack.

Re:SearXNG is a free internet metasearch engine by Fnoord on Friday April 05, 2024 @07:17PM (#39948418)
It is a metasearch engine. So it uses other search engines. The point is to let multiple use it, so that Google et al. does not know who's using their service. Ie. it is a gloried proxy.

Honestly, I just use Kagi. Though I need to find some way to limit my searches to 300 per month.

Re:SearXNG is a free internet metasearch engine by ranger_danger on Friday April 05, 2024 @07:50PM (#39948652)
that does not negate what OP said. your IP will still get blocked very quickly.

although existing searx instances have been run for years and they don't seem to be dropping like flies...

Re:SearXNG is a free internet metasearch engine by lannisterstark on Friday April 05, 2024 @11:07PM (#39949662)
Well. I host a public instance. IP is still not blocked. YMMV.
Re:SearXNG is a free internet metasearch engine by wkat4242 on Friday April 05, 2024 @11:07PM (#39949663)
Isn't Kagi also really a delegator? I've heard they delegate to brave among others.
Re:SearXNG is a free internet metasearch engine by HeatrayEnjoyer on Friday April 05, 2024 @07:15PM (#39948407)
Your IP address will get burned
Re:SearXNG is a free internet metasearch engine by baobun on Friday April 05, 2024 @08:51PM (#39949008)
Only if you expose it publically without auth while routing queries through your residential connection, which is not an advised configuration.

For personal use, you can run it directly on your machine or access over VPN. Queries to upstream search engines can be forwarded over proxies or VPNs as you see fit. Some work fine over tor and some can go over commercial or DIY tunnels.

Re:SearXNG is a free internet metasearch engine by ttt3ts on Friday April 05, 2024 @09:49PM (#39949316)
To add, I have been running instance for years for family and friends. I run it behind a nginix basic auth with a config that sets a forever cookie first time you login. Really simple. Another good option is cloud flare zero trust.
Re:SearXNG is a free internet metasearch engine by nickburns on Friday April 05, 2024 @09:59PM (#39949379)
how many total regular F&F users? do they ever ask you about logging? or is that beyond the scope of what most of them realize is happening?
Re:SearXNG is a free internet metasearch engine by ttt3ts on Saturday April 06, 2024 @09:15AM (#39952238)
A ~dozen. Several are technical and use it because it includes several private and paid engines on request.

Config is in a git repo I give access to if requested. One of the technical users modified it to keep pretty minimal logs. I guess they are trusting me to actually use that config but trust is pretty high in the group so not really an issue.

Re:SearXNG is a free internet metasearch engine by nickburns on Saturday April 06, 2024 @07:24PM (#39956604)
very interesting. sounds like an enviable group. thanks for your reply.
Re:SearXNG is a free internet metasearch engine by BeetleB on Saturday April 06, 2024 @12:36AM (#39949995)
I think this needs a lot more clarification than is provided in this thread.

If you run it locally, and only you use it, then you won't get blocked - a given search engine will see about the same number of requests as if you used it directly.

Add a few house members and you'll still be fine.

(I ran the original searx for a year or two locally - no issues at all).