You're thinking donkeys and horses. I believe a mule is the sterile result of that interbreeding.

Mules are mostly sterile, but not 100%. Wikipedia reports some cases where female mules have produced offspring.

Interbreeding can be ranked on a scale. The less related the groups are, the more problems there are with reproducing. Perhaps a hard boundary of "species" can be drawn where the probability of successful offspring approaches zero. However, that's difficult to measure in practice.