Monitor PasteBin for Your Email or Passwords

Re:Monitor PasteBin for Your Email or Passwords by twstdroot on Thursday February 06, 2014 @05:10PM (#7193274)

speaking from experience (https://github.com/bryanbrannigan/pastebin-parser) if you are just grabbing the pastes from the "latest" box you are missing a lot. To grab everything we actually had to create a distributed setup or else pastebin would start banning our IPs.

[ Parent | on HN ]

Re:Monitor PasteBin for Your Email or Passwords by jwcrux on Thursday February 06, 2014 @07:21PM (#7193905)

I got around this with @dumpmon by simply playing nice with Pastebin. I discovered what limits they liked/didn't like, and adjusted accordingly.

I take my entries from the archive as they are available, and I don't believe I ever miss any.

[ Parent | on HN ]

Re:Monitor PasteBin for Your Email or Passwords by shayanjm on Thursday February 06, 2014 @05:13PM (#7193286)

I ran into that issue myself. Pastebin throttling is real. I was playing around with the idea of actually using the socks5 proxies gathered through scraping in order to retain modularity (and eliminate the necessity of multi-IP set ups which could easily get pricey).

It would be tough because I would have to check the health of each proxy prior to use (so that I don't miss out on request windows), but still an interesting concept to consider.

[ Parent | on HN ]

Re:Monitor PasteBin for Your Email or Passwords by twstdroot on Thursday February 06, 2014 @05:15PM (#7193290)

cheap VPS boxes from lowendbox.com work well for this purpose. we also had problems just processing the queue on busy days.

[ Parent | on HN ]

Re:Monitor PasteBin for Your Email or Passwords by afreak on Thursday February 06, 2014 @05:26PM (#7193338)

Distribution is what I ended up doing. :)

[ Parent | on HN ]