Re:Headless mode in Firefox
by dboreham on Wednesday August 30, 2017 @01:48PM
(#15134299)
To add to the sibling posts:
Crawling the Web. Although you might think that you can crawl a web site by poking through the HTML looking for <a> tags and opening the URLs they contain; that stopped working well a long time ago. Modern sites are essentially complex GUI applications that run inside...a browser. You need either a browser, or some browser-equivalent thing therefore in order to run them and properly crawl them.
Re:Headless mode in Firefox
by camoby on Wednesday August 30, 2017 @02:49PM
(#15134831)
Isn’t this why these ‘headless’ solution have (or are actually built on) JavaScrict engines like V8 with the ability to also simulate any user interaction?
Crawling the Web. Although you might think that you can crawl a web site by poking through the HTML looking for <a> tags and opening the URLs they contain; that stopped working well a long time ago. Modern sites are essentially complex GUI applications that run inside...a browser. You need either a browser, or some browser-equivalent thing therefore in order to run them and properly crawl them.