If your pixels are a byte each and the data is 64-byte aligned in memory, you're still reading 640 * 2 + 128 * (480 - 2) = 62464 bytes from memory, not 1080. That's because you can't read less than a cache line, which happens to be almost always 64 bytes. Performance win could be a lot less than what you expected.