That's true, but unlike the DVD case it's so only because the GPL license explicitly grants you this right. If it didn't, you wouldn't be able to run it at all.
GPL is actually kind of a special case when it comes to licenses since it mostly cares about redistribution in order to implement its copyleft mechanism, but generally licenses are tools used to give users permission to use the software under specified conditions. Without a license (written or verbal), there's no permission to do anything whatsoever.
You may be confusing it with using a self-modified copy of code under GPL. Since GPL only cares about sharing the modified source with actual users, you won't have to share your modifications with anyone as long as you remain the only user of that modified work.
What happens if you do?
Likewise if I take some GPL software, I can do whatever I want with it to run it, including copying it into memory. I can't give a copy to anyone else (including derivative works such as a binary) without agreeing to the GPL.