> Using upvotes to indicate "I've replied to you" is also going to be drowned out by noise because a poster has no way to know who has upvoted therefore no way of distinguishing your "I've replied to you" upvote from a regular user who upvoted their reply because it was informative/useful.

While I fully agree, it works well enough as is :)