That’s all it is.
Basically you have a DAG just like you think. But git needs some way to label some of the nodes in that DAG. It calls those labels “references” or refs, it records them under .git/refs/, and there’s basically just three kinds: local branches (refs/heads/<branch_name>), tags (refs/tags/<tag_name>) and remote tracking branches (refs/remotes/<remote_name>/<branch_name>). There’s also the reflog which is a change log of the local branch refs, and a special log for HEAD which keeps track as you switch which branch is currently checked out.
Note: if you start poking around under .git (and you should in a toy repo!), you might not find exactly these files. For performance reasons, git sometimes combines the refs into a single file called packed-refs.
The only thing you really need to internalize is that all git operations are entirely local on your machine, except clone/fetch/pull/push.
(Another useful thing to internalize is that git pull is really just git fetch followed by git merge. git fetch is the part that involves network activity, while git merge is entirely local on your machine. I can't remember the last time I've used git pull, actually...)
So when you're doing git log origin/master, you're not looking at the current state of the master branch on a remote machine. Instead, you're looking at what the state was the last time you did a fetch/pull/push.
This is extremely useful for being able to work remotely.