Re:Show HN: Browsh – A modern, text-based browser
by jacquesm on Tuesday July 10, 2018 @07:01AM
(#17497003)
The web 1995-2005 or thereabout will be the best documented period of humanity and it is saddening to see how payload has been replaced by cruft to the point that we might as well have remote PDF viewers.
So much for 'the semantic web' I guess.
Re:Show HN: Browsh – A modern, text-based browser
by tombh on Wednesday July 11, 2018 @02:58PM
(#17509363)
This precisely the sort of thing that Browsh can remedy!
Re:Show HN: Browsh – A modern, text-based browser
by tombh on Tuesday July 10, 2018 @04:54AM
(#17496364)
I've thought about this too, because https://text.brow.sh certainly does make scraping websites easy, you just need to `curl` and `grep`. But the trouble is, as I mentioned elsewhere in the thread, Browsh's text renderer is more of an art than a science, it doesn't guarantee consistent results for every render. Whereas of course downloading HTML and using something like Beautiful Soup does give a much better guarantee of consistency.
I'd considered using a headless browser setup but I never got 'round to it.
If you structured this as a paid service where I could make some number of requests for some amount of money and get back basically what you've got now in monochrome mode, I'd probably sign up pretty quick.
This is sort of relevant to some other recent HN discussions around the trouble with large-scale crawling and the trouble with so much of it being restricted to Google now, so I don't think I'm the only one.