I try to worry much more about what works rather than labels.
If you stuff all your functionality and data transformations into class definitions, your class definitions will be full of dependencies. Now all the fundamental elements of your program depend on each other and can't be separated.
It's like trying to pile up concrete until it forms a cave instead of laying it down to make the floor and building on top of it.
What am I missing?
If classes are only used as data structures, and everything is done through (presumably pure) utility methods, it sounds like you're writing procedural code in an OOP language.
That's not inherently a bad thing, but OOP provides benefits and you may be making a trade-off without thinking about it.