Tree nuts are another common allergen, along with milk, eggs, fish, shellfish, wheat, and soy[1]

Presumably peanuts and tree nuts are more commonly singled out due to the severity of the potential reaction, as well as the likelihood of accidental exposure (ground or powdered nuts tend to spread easily and unintentionally compared to the other allergens)

1: https://www.cdc.gov/healthyschools/foodallergies/index.htm

The linked CDC article could be better on this: Tree nuts, peanuts (legumes), and seeds (sesame, etc.) can be actual allergens. I've never heard of a single case of milk or wheat allergy in an "allergic reaction" standpoint.