Scraping and indexing 1.2B emails for under $200

Posted by jordiee on Thursday March 12, 2020 @11:01AM
from the jordanpiepkow.com dept.

https://jordanpiepkow.com/scraping-and-indexing-1-2-billion-emails-for-under-200/

( 6 comments | 30 points | on HN )

Re:Scraping and indexing 1.2B emails for under $200 by tdeck on Thursday March 12, 2020 @05:38PM (#22562062)

Is nobody going to mention how this is a bad thing intended for sending spam? Guess I'll have to be that person then.

[ Parent | on HN ]

Re:Scraping and indexing 1.2B emails for under $200 by ad404b8a372f2b9 on Thursday March 12, 2020 @02:43PM (#22560231)

Took me a while to understand they were scraping email addresses and not actual emails.

[ Parent | on HN ]

Re:Scraping and indexing 1.2B emails for under $200 by hbcondo714 on Thursday March 12, 2020 @02:20PM (#22559933)

Wow, the author really calls out his competition! There are also parts 2 and 3 to this article that discusses using Rust and Postgres for their solution.

[ Parent | on HN ]

Re:Scraping and indexing 1.2B emails for under $200 by natmaka on Friday March 13, 2020 @03:56AM (#22565535)

Help rid the world of spam! Project Honey Pot is our friend. https://www.projecthoneypot.org/

[ Parent | on HN ]

Re:Scraping and indexing 1.2B emails for under $200 by thomas536 on Thursday March 12, 2020 @09:22PM (#22563862)

I must be missing something because 6.5 days * $21/day = $136.5

"""

The entire process now took 6.5 days and cost $21/day. Our total cost all said and done was $115!

"""

[ Parent | on HN ]

Re:Scraping and indexing 1.2B emails for under $200 by slowhand09 on Thursday March 12, 2020 @01:06PM (#22558995)

Nice writeup.

[ Parent | on HN ]