You could include "Closes #123" or "Merges #123" or "Fixes #123" in the commit message of the merge commit, and GitHub will link it, and currently close it (it doesn't actually consider it merged, but it at least auto-closes and links)