If you can create a customized font file that is tiny and only has the icons you actually use, sure.

But the "wrong tool for the job" point still stands, users with custom fonts disabled (for performance or accessibility reasons) will not be able to see your icons.