Imagine a web server that didn't wipe plaintext passwords or encryption keys from its memory after finishing with them. If the web server was remotely exploitable then it could be possible to obtain the contents of the memory of that process remotely, thus possibly leaking passwords or other sensitive information of other people that have connected to that web server at some point in the past.
> how can you be sure that someone didn't access it before your memset() call?
True, but there's a difference between having a very small window of opportunity where the data could be obtained via a remote exploit, and leaving the window wide open for possibly endless period of time.
Obviously you shouldn't have any remote exploits in the code in the first place, but it's good practice for secure programming to keep the sensitive information in memory for as short a period as possible just in case there is something that you aren't aware of.
Bear in mind that other normal user processes can only access the memory once it has been returned to the OS. Therefore, it means you have to have a malicious super user to access your keys whilst the program is running.
Of course the memory will be available to other processes because we don't have infinite supply of memory. Stuff like mlock(2) can prevent paging to disk, and there are probably other countermeasures too which I'm not aware of.
Using something like mlock(), supposedly.
If the memory of the process is available to other processes after it finishes (or while it's running), isn't this already a lost game? I.e., how can you be sure that this particular chunk of memory wasn't cached on disk at some point? how can you be sure that someone didn't access it before your memset() call?