I am similarly confused as to why docker needs a daemon. For what's provided in the API, I don't see the need...it could all happen in the client.

Contrast with LXD. It has a daemon as well, but it tracks containers and state across multiple hosts, so the need is obvious. The single host LXC doesn't have a daemon.