Affirmative action is not generally about giving people money, it's about giving people opportunities. The idea is something like, you set a lower bar for people from groups that have been historically discriminated against.

When we have programs that are specifically designed to help people monetarily, such as financial aid at colleges or welfare, they consider people's economic situation but not their race.