HEAD is a pointer. It points either indirectly to a commit through the name of a branch, or directly to a commit when you've checked out something other than a local branch (so called detached HEAD). In this way it acts both like a symlink and like a hard link. So:
HEAD -> branch name -> commit ID; or
HEAD -> commit ID.
Its literal representation is a file named HEAD whose contents are either the name of a branch or a commit ID.
A tag is also a pointer. It may also point either indirectly to a commit through an annotated tag object or directly to a commit (so called lightweight tag). So:
tag -> tag object -> commit ID; or
tag -> commit ID.
Its literal representation is a file under refs/tags named after the tag and whose contents are the ID of the tag or commit object.
(Ignoring optimizations such as packed refs and the exact formatting of ref files. See Git book and man pages for details, or poke around your local .git directory. It's very explorable.)
I hope Linus is actually remembered for this more than the creation of Linux.