If you gave the owners of some website permission to star repos on your behalf, then you are trusting them; you are still the one responsible for your endorsements. Whether endorsing the wrong repo ought to ban worthy is definitely up for debate. But the question of whether you endorsed manually or delegated it seems unrelated.