bndr

I disagree with the article a lot. Solving problems because easier yes, the size of the problems became bigger. What I can do now with an LLM is magnitudes larger than I could've done alone.

Some links lead to gambling landing pages e.g. backery

Users sign up for my service.

Please elaborate, why exactly is it antisocial? Because Cloudflare decides who can or cant access a users website? When they specifically signed up for my service.

They're mostly non-technical/marketing people, but yes that would be a solution. I try to solve the issue "behind the scenes" so for them it "just works", but that means building all of these extra measures.

I run a small startup called SEOJuice, where I need to crawl a lot of pages all the time, and I can say that the biggest issue with crawling is the blocking part and how much you need to invest to circumvent Cloudflare and similar, just to get access to any website. The bandwith and storage are the smallest cost factor.

Even though, in my case, users add their own domains, it's still took me quite a bit of time to reach 99% chance to crawl a website — with a mix of residential proxies, captcha solvers, rotating user-agents, stealth chrome binaries, otherwise I would get 403 immediately with no HTML being served.

Adding a bit of context as well: This started out as a internal linking tool, but grew into something more based on the customer feedback — the database has now reached about 10TB of data about keywords, pages, AI responses etc, where I know who was ranking where and why.

And I'm trying to offer this "data advantage" to website owners, so they can grow, and also this is something that will be hard to replicate (at least quickly) with AI.

I've been working on the same tool since 2024 where I thought it might be a good time to build a tool for all the people who will build their own tools, eventually they will need to market it.

So I built a SEO/GEO Automation Tool for Small to Mid-Size Businesses who don't have a full-time team for that. [0]

The goal is to provide teams visibility across all the channels — Search and AI and give them the tools needed to outrank their competition. So far so good, the fully bootstrapped venture has grown over the last year and I've built quite a few big features — sophisticated audit system, AI Responses Monitoring, Crawler Analytics, Competitors Monitoring etc.

[0] https://seojuice.io

Oh wow, that was painful to read, I especially liked this analysis part:

> Different naming conventions (DW_OP_* vs DW_op_*)

Hey HN!

I’m working on SEOJuice [1], an automated tool for internal linking and on-page SEO optimizations. It's designed to make life a little easier for indie founders and small business owners who don’t have time to dig deep into SEO.

So far, I’ve managed to scale it to $3,000 MRR, and recently made the move from the cloud to Hetzner, which has been a game-changer for cost efficiency. We’re running across multiple servers now, and handling everything from link analysis to on-page updates with a bit more control.

The journey’s been a mix of hands-on coding (and a lot of coffee) and constant optimization. It’s been challenging but incredibly fun to see how much can be automated without compromising on quality.

Happy to chat more about the tech stack or any of the growth pains if anyone’s interested!

[1] https://seojuice.io

Oh wow, my package on the front page again. Glad that it's still being used.

This was written 10 years ago when I was struggling with pulling and installing projects that didn't have any requirements.txt. It was frustrating and time-consuming to get everything up and running, so I decided to fix it, apparently many other developers had the same issue.

[Update]: Though I do think the package is already at a level where it does one thing and it does it good. I'm still looking for maintainers to improve it and move it forward.

Hey everyone, I know HN community is very polarizing, and the discussions here are always great to read through as both sides are always eager to prove the other wrong. I think we need more of that in the community. People not being afraid to disagree.

I'm really curious to hear your thoughts and experiences.

Looking for feedback.

Cheers :)

If you see any bugs, please let me know. The product is quite fresh.

Why not just retrain the python team to another language? I mean, software engineers are not really language specific, they can learn other languages if needed.