Therefore, in implementations such as OpenPGP/PGP, a new random symmetric key is generated with which the plain text will be encrypted, for example using AES as you suggested. Afterwards, this symmetric key will be encrypted with the public key of the recipient and attached to the cipher text.
When sending a document to multiple recipients, you only need to encrypt the symmetric key multiple times. The symmetric key is usually short compared to the rest of the document.
So if you are only encrypting for a single recipient you could encrypt using the public key directly. However, it would still need more computing power. This might be negligible on modern Desktop computers with lots of computing power and even hardware crypto support, but for embedded devices such as your smartphone it's still an important factor.
PS: Of course, there is more to OpenPGP/PGP such as the Web of Trust and digital signatures for verification of the sender of a document.