Re:Old Internet Files
by abritinthebay on Monday June 25, 2018 @07:41PM
(#17396789)
Web Spiders have been around since ‘94 and the spider terminology for mapping networks is even older so they’d likely get it.
[ Parent
| on HN ]
Oh geez. Webcrawler, and robots.txt. There was no SEO, just "don't overwhelm my modem / ISDN connection / T1". It's been a while!
Re:Old Internet Files
by abritinthebay on Wednesday June 27, 2018 @01:31AM
(#17405993)
A lot more interesting protocols too! I miss gopher & finger