If a historically marginalized group feels that you are appropriating their culture, the proper response is not to learn their history and then tell them that since you read a book it's now OK. It's to stop.
I do think there are some areas where most people can agree that use of another culture's artifacts is in poor taste and not generally acceptable. A very specific example would be that members of a culture that conquered another culture shouldn't wear clothing or jewelry that was looted from the conquered people. I think most people would agree that that crosses a line. Same for costumes that trivialize or mock another culture, like children playing cowboys and "Indians" in the 20th century, or minstrel shows. But outside of that core, I think it turns into a grey area very quickly.
It's a really tough problem to find a good answer to, and IMO it's impossible to find one that makes everyone happy because of that diversity of opinions in nearly every culture.
For example: Eminem is respected as a rap artist because he apprenticed himself to the culture and celebrates it. He didn’t just try to make money off rap and learn a bit of history about it to justify doing so.