What were they emailing themselves, IP?

All sorts of things. Sometimes it's code or documentation/guides they wrote that they believe they're entitled to... Sometimes it's personal things they kept on their work computer/account.... Sometimes it's contact lists... Sometimes it's strategy docs they authored... Sometimes it's password files (which often mix personal and work creds). Sometimes it's whatever-they-can-think-of that might be of value on the share drive...

It's just so incredibly common that I don't understand why any employer doesn't terminate access the instant someone is let go. When people they're in a state of stress just grab whatever they can think of.

What about emails that they might need to bring up in court?

What's the general way for an employee to retain access to those?

If they file a case in court, they are entitled subpoena documents that might support their case, including emails. There is an established procedure for that and company legal/compliance departments have legal-hold policies to freeze anything that might be relevant for a pending legal case.

Trying to forcibly steal those documents/emails is a wonderful way to being charged with a computer crime if it's serious enough. ...but lesser consequences include public lawsuits that impact your career prospects (we did this to one high ranking employee who tried to email herself powerpoint strategy presentations she made and copied to a USB). ...but at the very least, having an attempted theft of documents on record will get any future allegations they might make against the company much easier to throw out.