> by 4th grade I'd be teaching about slavery, racism, etc, and the kids would be then learning about a concept that was utterly nonsensical to the non-racism-indoctrinated kids. So I'm wondering what would happen if I never taught about racism at all.

Assuming these kids are in the U.S., then you omitting the topic from their 4th grade class would only omit a tiny portion of what they encounter in their lives. If my 4th grade teacher omitted something from history class, I don't think it's possible that I've never learned about it (assuming it has some significance).