You can control expansion of heredoc content by quoting the delimiter, like:

  ssh [email protected] <<'HERE'
    echo $USER
  HERE

Per POSIX, "If any part of word is quoted, the delimiter shall be formed by performing quote removal on word, and the here-document lines shall not be expanded. Otherwise, the delimiter shall be the word itself." "If no part of word is quoted, all lines of the here-document shall be expanded for parameter expansion, command substitution, and arithmetic expansion."