Taste and spiciness. Yellow mustard seeds are milder and black mustard seeds are spicier. Brown seeds are usually in the middle. Most recipes usually call for specific amounts of each, once you know what you like, you can adjust them.