Re:Show HN: A Python Spider System with Web UI
by binux on Monday November 17, 2014 @11:23AM
(#8618847)
The fetcher fit you already...
[ Parent
| on HN ]
Re:Show HN: A Python Spider System with Web UI
by maratc on Monday November 17, 2014 @11:42AM
(#8618965)
You are running
phantomjs phantomjs_fetcher.js
and using it as proxy? The setup instructions are a bit unclear on this.
Re:Show HN: A Python Spider System with Web UI
by binux on Monday November 17, 2014 @11:52AM
(#8619024)
I want to make it a http proxy in the beginning. But I found it hard to do so. Then I post every to it, but haven't change the name.
But it works like a proxy, that any request with `fetch_type == 'js'` would be fetched through phantomjs and the response back to tornado_fetcher.