In many (but not all) Linux package distributions, malware is filtered out by maintainers. In other package distributions, maintainers are omitted, because «it's too hard». Instead, everybody protects it's own system alone.

Even if your ssh-key will not be stolen when project is built, it still can happen when malware will be shipped to your client. How you protect your client?