There are VIA motherboards with build-in CPU and GPU for one (I had one of these for a small NAS, which could have been used for a very lightweight computer as well).

Raspberry Pi and stuff like that use other brands of GPU as well.