Re:HBO Max accidentally sent an integration email test to users
by wiredfool on Friday June 18, 2021 @07:25AM
(#27549296)
When I worked on email systems, my worst nightmare was a Sorcerers Apprentice sort of problem.
Re:HBO Max accidentally sent an integration email test to users
by yawnxyz on Friday June 18, 2021 @07:55AM
(#27549433)
oh man I just wrote a piece of code that lets me write any markdown, push a button and it just sends it out to our ~1-2000 users.
An hour later I just commented it all out, and wrote a note to myself: "if you need this, uncomment and push back to stage". Just having that code even sitting around makes me nervous
Re:HBO Max accidentally sent an integration email test to users
by samus on Friday June 18, 2021 @09:21AM
(#27549980)
We introduced an allowlist on all our testing and staging environments to ensure that only certain recipients can get email. We also make sure that no email address in these databases would work, unless we really want to send to it.
My script was supposed to try to grab batches of 1000 customers and keep looping until it ran out of customers (signaled by having retrieved less than 1000 customers in my last request for the next batch of 1000 customers).
My script was missing the offset part of the query, so after we hit 1000 users, it just kept looping, sending the same email over and over to our first 1000 users.
I felt so bad that day. From then on, sending out emails was this whole huge process that involved queuing them all and then having like 6 people review to make sure we didn't mess it up.