News for hackers. Stuff that matters.
Re:Show HN: Business.txt - Standard Proposal by quadhome on Monday August 27, 2012 @09:39AM (#4438716)
You'd rather a robot hit your frontpage— and by association, your application- than hit your 404 static page?

Instead of generating errors from 404s (a lot of noise), try generating errors from the same repeated 404s.

Re:Show HN: Business.txt - Standard Proposal by pilif on Monday August 27, 2012 @10:18AM (#4438900)
The robot will very likely hit my frontpage anyways, so I'd rather have them look (and not find) for a meta tag there than to produce an additional 404 error which will, depending on the content of said page, waste a considerable amount of bandwidth.

Besides, my frontpage is either heavily fragment- or just page-cached anyways - especially for anonymous user - so it can be served directly from RAM for all intents and purposes, so a hit to the frontpage is really, really cheap.

Re:Show HN: Business.txt - Standard Proposal by ricardobeat on Monday August 27, 2012 @11:15PM (#4441953)
Cheaper than a hit to a static text file? It's cached in RAM too.
Re:Show HN: Business.txt - Standard Proposal by egypturnash on Monday August 27, 2012 @11:59AM (#4439395)
> an additional 404 error which will, depending on the content of said page, waste a considerable amount of bandwidth.

Because your 404 should be one of the heaviest pages on your site, full of graphics and surprises. Why not make a full-featured game specifically for your 404s?

Re:Show HN: Business.txt - Standard Proposal by icebraining on Tuesday August 28, 2012 @04:20AM (#4442657)
Crawlers don't (usually) download graphics, much less from 404 pages. They aren't regular browsers.
Re:Show HN: Business.txt - Standard Proposal by wedtm on Monday August 27, 2012 @06:33PM (#4441255)
thebest404pageever.com
Re:Show HN: Business.txt - Standard Proposal by jewel on Monday August 27, 2012 @12:23PM (#4439524)
We reduce the noise by only looking at 404s with referrers.
Re:Show HN: Business.txt - Standard Proposal by alttag on Monday August 27, 2012 @07:44PM (#4441429)
That will cause you to miss some behavior if internal URLs ever change that users bookmarked or remember.

As an example, a TV station site I follow had a /fullepisodes/ basename page which wasn't updating in sync with their show pages, apparently pulling from a different data store. After about six months of declining updates to that page, they eventually re-vamped it with a more-correct data source. I like to think it happened because I and others kept browsing directly to the easily memorable URL