The issue is the timing. If you defrost in the fridge, it will eventually reach ambient temperature, but that's still safe. If you defrost on the counter, it will briefly be cold-but-defrosted, but rapidly progress to room-temperature-petri-dish.