That's completely true because there are many aspects to what is "my group" and what isn't, but the key point is, people naturally care about their group more than they care about strangers. Thinking in terms of genetics provides a simple model that's good enough to explain a lot of phenomena. But yes, if you want to go deeper, you need to consider other factors - at first glance it seems like "culture" is the most important one.