Re:Show HN: Headless Chrome Crawler
by bryanrasmussen on Thursday February 22, 2018 @04:03PM
(#16441368)
yes I've used puppeteer, but I couldn't see where it exposed the page object, but looking closer I saw
HCCrawler.launch({
// Function to be evaluated in browsers
evaluatePage: (() => ({
title: $('title').text(),
})),
// Function to be called with evaluated results from browsers
onSuccess: (result => {
console.log(result);
}),
})
which doesn't look that good for my other worry, that you come to a dynamic JS built up page, load the DOM and evaluate in some milliseconds and then that DOM is changed after you've done your incorrect evaluation.
HCCrawler.launch({ // Function to be evaluated in browsers evaluatePage: (() => ({ title: $('title').text(), })), // Function to be called with evaluated results from browsers onSuccess: (result => { console.log(result); }), })
which doesn't look that good for my other worry, that you come to a dynamic JS built up page, load the DOM and evaluate in some milliseconds and then that DOM is changed after you've done your incorrect evaluation.