News for hackers. Stuff that matters.
Re:Gmail blows up e-mail marketing by caching all images on ... by surrealize on Thursday December 12, 2013 @03:28PM (#6896980)
Google could retrieve and cache a random sample of the images and hash them. Then they could note which image links go to the same images. And they could identify image links by noting that a bunch of emails have the same structure except for one image link that's different for every recipient.

So I think they actually could take a pretty good stab a deduping images with unique tracking URLs. It might not be perfect, but even if it works 95% of the time they could still kill the profitability of the unique tracking image technique.

Re:Gmail blows up e-mail marketing by caching all images on ... by codeflo on Thursday December 12, 2013 @03:30PM (#6896993)
Easy: Make the "Dear <username>" part of the email an image. Boom, deduplication fooled.

(Also, they don't currently seem to do anything like you're suggesting.)

Re:Gmail blows up e-mail marketing by caching all images on ... by Nacraile on Thursday December 12, 2013 @03:37PM (#6897069)
What if google just immediately fetches images for all received messages, regardless of deliverability? They can dedupe wherever possible, but the sender still ends up with no information about whether the message was delivered, much less read.
Re:Gmail blows up e-mail marketing by caching all images on ... by Silhouette on Thursday December 12, 2013 @06:18PM (#6897977)
What if google just immediately fetches images for all received messages, regardless of deliverability?

I'm trying to work out whether this is more useful as a way to get Google to DoS themselves or as a way to get them to DoS arbitrary web sites of others. Either way, isn't this a gift to trouble-makers?

Of course Google would probably develop an automated defence against such attacks quickly if they happened in practice, but it seems any such defence would necessarily involve not caching all the images in advance, which would defeat the original point.

Re:Gmail blows up e-mail marketing by caching all images on ... by Nacraile on Thursday December 12, 2013 @06:58PM (#6898169)
I'm fairly sure sending an email is more expensive than sending a GET, so it should be more effective for an attacker to make the requests directly than trying to use this to get google to proxy an attack.

I also strongly suspect that google's crawling infrastructure is more than capable of fetching a bunch of images for every single message gmail receives.

But even if I'm wrong about the above, google is perfectly capable of throttling their fetching to mitigate. (The problem really ends up looking an awful lot like crawling the internet, which is an area that google seems to have a bit of experience)

Re:Gmail blows up e-mail marketing by caching all images on ... by Silhouette on Thursday December 12, 2013 @07:16PM (#6898262)
In reality, I'd be less worried for Google and more worried for whoever is hosting moderately large images that get linkjacked in numerous variations (http://www.example.com/largeimg.png?randomnumber=72435).

Google can't tell, a priori, whether or not a series of similar e-mails sent to many thousands of people with Google Mail addresses and containing similar but different image links like the above is a genuine mail going out to someone's list or a DDoS of www.example.com in which Google is about to become an unwitting participant.

By the time they've worked out whichever trick is being used this time (in the same way that they adapt to changing black hat SEO tactics, but probably only make major changes every few months) it's not hard to see a hostile party busting the bandwidth cap for anyone on a basic, low-volume hosting plan.

Re:Gmail blows up e-mail marketing by caching all images on ... by _greim_ on Friday December 13, 2013 @01:42AM (#6899397)
Why involve Google? Aren't sites on basic, low-volume hosting plans easy to knock over, without resorting to DDoS tactics? And if you're trying to knock over bigger sites, it doesn't seem like Google would make a very good DDoS platform in any case, since the requests would be originating from a relatively small range of IPs that a bigger site could just ban. Presumably the only reason they wouldn't want to ban the requests is if they're actually the ones sending the emails in the first place, so the problem sort of solves itself.
Re:Gmail blows up e-mail marketing by caching all images on ... by JimDabell on Friday December 13, 2013 @08:28AM (#6900358)
This is an old problem with an old solution. If you have an expensive-to-generate resource that you don't want automatically retrieved en masse, you use robots.txt to deny access to it.
Re:Gmail blows up e-mail marketing by caching all images on ... by jaxn on Thursday December 12, 2013 @04:01PM (#6897231)
AND it could create a dis-incentive to load up an email with unique images since as soon as you send the email out all of those gmail addresses are coming right back at your server to request the images.
Re:Gmail blows up e-mail marketing by caching all images on ... by jblow on Thursday December 12, 2013 @04:42PM (#6897468)
And then you can cause Google to DDoS someone else's site by sending out spam containing lots of image URLs.
Re:Gmail blows up e-mail marketing by caching all images on ... by xur17 on Thursday December 12, 2013 @08:55PM (#6898603)
You can somewhat do this with the current system. I had no problems sending an email with 10 10mb images. Google happily fetched all 10 of the images off my server.

Not sure if they limit it at some point, but if a server accepts urls such as:

http://i.imgur.com/9Y5FDz7.jpg http://i.imgur.com/9Y5FDz7.jpg?1 http://i.imgur.com/9Y5FDz7.jpg?2 etc...

Google would fetch each separately. Send this out to a bunch of people, and it seems problematic. I'm going to be optimistic, and assume they built in some sort of limiting, but who knows.

Re:Gmail blows up e-mail marketing by caching all images on ... by prostoalex on Thursday December 12, 2013 @06:27PM (#6898013)
Users would have to have previously agreed to "Always load images from domain.com"
Re:Gmail blows up e-mail marketing by caching all images on ... by vidarh on Thursday December 12, 2013 @06:53PM (#6898149)
The cost of a new domain per e-mail campaign would be trivial.

(and "normal users" do click the show images links)

Re:Gmail blows up e-mail marketing by caching all images on ... by vidarh on Thursday December 12, 2013 @06:39PM (#6898062)
We manage opt-in mailing lists for customers of restaurant chains, and for well timed, well-targeted campaigns they get open rates in the 30%-40% range with the majority of opens within 15-30 minutes of the send anyway. It takes more resources for us to handle the outbound mail load than to handle the inbound image requests, as for the image requests the url's contains enough info to do a trivial regexp based rewrite and fetch the images from a cache. I don't think handling a 100% open rate as soon as the mail was delivered would even remotely be a challenge.
Re:Gmail blows up e-mail marketing by caching all images on ... by jaxn on Sunday December 15, 2013 @03:53PM (#6910800)
Just curious, how big are those lists? There is a big difference in a restaurant with 10,000 customers and an e-commerce site with 3,000,000.
Re:Gmail blows up e-mail marketing by caching all images on ... by vidarh on Thursday December 12, 2013 @06:48PM (#6898123)
Yeah, good luck with that. Instantly, every hacker in the world would use Gmail as a DDoS amplification tool.
Re:Gmail blows up e-mail marketing by caching all images on ... by polarix on Thursday December 12, 2013 @04:11PM (#6897313)
This is the correct answer. If google does not do this, well, we know whose side they're on.
Re:Gmail blows up e-mail marketing by caching all images on ... by grahamburger on Thursday December 12, 2013 @06:47PM (#6898110)
PG recommended this method years ago for fighting spam. Puts a load on the spammers. Not sure if anyone has ever tried it at scale.
Re:Gmail blows up e-mail marketing by caching all images on ... by surrealize on Thursday December 12, 2013 @03:33PM (#6897025)
With google's machine learning brain trust, I think they could still do a pretty good job deduping. Maybe not perfect, but I'd bet on them to win an arms race.

Edit: ah, codeflo and EGreg are right. I was just thinking about the task of determining that the images serve the same role in each message (which I'm sure google could do a good job of). But (as they point out) in the "Dear <user>" case they'll still have to show the right image to the right user. Although, as Nacraile and jaxn say, if they load all those images eagerly they'll remove the value of those unique tracking images, and impose a cost on the sender.

Re:Gmail blows up e-mail marketing by caching all images on ... by codeflo on Thursday December 12, 2013 @03:49PM (#6897149)
There's a huge difference between something that they might do, in theory, at some point, using vaguely magic machine learning technology, and what that they are currently doing, right now, to address the privacy concerns over a change that they already rolled out to millions of users.
Re:Gmail blows up e-mail marketing by caching all images on ... by EGreg on Thursday December 12, 2013 @03:57PM (#6897200)
Are you saying Google will try to guess and reconstruct "Dear Marge" in the same font as "Dear John" instead of requesting both from the origin server?
Re:Gmail blows up e-mail marketing by caching all images on ... by backlava on Thursday December 12, 2013 @09:59PM (#6898812)
Getting the font right is the easy part. Figuring out the formerly occluded pixels in the background image is the hard part.
Re:Gmail blows up e-mail marketing by caching all images on ... by ceejayoz on Thursday December 12, 2013 @04:23PM (#6897381)
> But (as they point out) in the "Dear <user>" case they'll still have to show the right image to the right user.

They could do a "This sender appears to be attempting to track you. We have disabled images as a precaution. Click here to load."

Scary enough to the average user it'd probably kill the the technique very quickly.

Re:Gmail blows up e-mail marketing by caching all images on ... by Silhouette on Thursday December 12, 2013 @06:15PM (#6897961)
Scary enough to the average user it'd probably kill the the technique very quickly.

I'm not sure about that. It's nothing that desktop clients such as Thunderbird haven't been doing for years. I don't see any remote images in any e-mail until I click to say load them, and this works in much the same way that plug-in elements like Flash and Java are now click-to-show in various browsers. Numerous marketers and mailing list services still use the technique to track an approximation of reader numbers, though.

Re:Gmail blows up e-mail marketing by caching all images on ... by rspeer on Saturday December 14, 2013 @12:34AM (#6904771)
Nah, nobody pays attention to warnings attached to Gmail messages after having seen so many of them.

A particular example of crying wolf that comes to mind is the yellow box that says, "HEY! THIS SENDER ISN'T WHO THEY SAY THEY ARE!", which usually means that someone just forwards their .edu address to Gmail.

Re:Gmail blows up e-mail marketing by caching all images on ... by scoot on Thursday December 12, 2013 @09:11PM (#6898660)
Or just cache unique images on delivery...
Re:Gmail blows up e-mail marketing by caching all images on ... by anatari on Friday December 13, 2013 @02:49AM (#6899527)
Although I'm sure it's possible to fool their image hashing algorithm, I doubt this will. Image hashing algorithms are designed to be resistant to small changes in the image and more advanced ones can generate hashes that determine how similar one image is to another. I haven't tried this, but you can probably see a proof of this using google image search. Add some text to an image, and see if Google image search can find the original.
Re:Gmail blows up e-mail marketing by caching all images on ... by webjprgm on Friday December 13, 2013 @09:39AM (#6900663)
Procedurally generated fractal backgrounds with random seed, that might work?

Anyway, I think it would be perfectly fine if Google matched up emails with similar content and where there was one image that was unique for everybody just remove it, maybe with a note to the user in that case.

Do NOT have a "click to load images" button. If users can't ever see them then it completely destroys spammers' ability to use them even for a rough sampling.

I would love to see spam as an advertising method be completely destroyed. It won't be, because even without tracking it is still easy and useful to spam out lots of ads, but this would help.

Re:Gmail blows up e-mail marketing by caching all images on ... by nly on Thursday December 12, 2013 @09:30PM (#6898722)
And Google could just mark any mail from sources who pull these shenanigans as spam
Re:Gmail blows up e-mail marketing by caching all images on ... by webjprgm on Friday December 13, 2013 @09:41AM (#6900672)
Yup. Though I suspect my university's alumni newsletter also has tracking images in it. I haven't checked, but if I were them I'd use a tracking image.
Re:Gmail blows up e-mail marketing by caching all images on ... by jpadkins on Friday December 13, 2013 @09:56AM (#6900748)
they dont have to even edit the image. they can just put a tracking id in the image metadata.
Re:Gmail blows up e-mail marketing by caching all images on ... by pramodliv1 on Thursday December 12, 2013 @03:47PM (#6897136)
What if the images are slightly modified for each user? A mixture of image processing and hashing can be used for deduping though.
Re:Gmail blows up e-mail marketing by caching all images on ... by vidarh on Thursday December 12, 2013 @06:43PM (#6898082)
E-mail marketing companies would make a fortune coming up with "image composition plugins" for their systems. E.g. manipulating background patterns; alter the positioning of image elements; change text positioning, size, fonts; change objects in the images.

It'd be a gold mine.

Re:Gmail blows up e-mail marketing by caching all images on ... by jtheory on Thursday December 12, 2013 @10:07PM (#6898834)
I'm pretty sure this already exists -- I've gotten spam with obviously random noise, lines, etc. added to the background.

My assumption is that it's making the images unique.