You're right of course, but there is a reason the colloquialism takes this form as opposed to another. Colloquialisms, the intelligible ones at least, sublimate some underlying element of culture in order to communicate sensibly and easily.

This very idea of stretching logic to extremes without consideration for real world effects is very much a foundational element in programmer culture. It's born out of rationalism and certain elements of enlightenment thinking. However, this pure (im)practical reason is often applied in all circumstances irrespective of the context. After all, it's possible to reason about a person's absence without also implying the worst, and this kindness is valuable. If SV could somehow change its foundational thinking from reason + corrosive business practices to reason + kindness we'd all be better off.