You're right that a lot of these stories are retold as propaganda to justify conquest but:

> was it hard to just back off and accept that people whoever they are should keep their land ?

It was only "theirs" by the same process of brutal conquest that the Spaniards later turned on them.