In other situations, this approach would work well as long as what I was working on was fresh in my mind. However, when I had to come back to some code and extend its behaviour it would be back to digging through it to either understand the structure of the data... or understand the structure of the fancy abstraction I had built up on top of the data. It just didn't seem to be very time efficient in practice for the things I was trying to build.
I feel the heart of the issue is surrounding a similar problem that message-passing set out to solve, perhaps encapsulation? You set up some boundaries to a thing and define some contracts for interacting with it, and in return you are free to separate the implementation of that from the consumer of that data.
I worry that when people program in Clojure, they forget all the discipline that was embedded in some of the better parts of the previous language/paradigms they came from and mistake that as liberation.
If you have entities in your domain, presenting clean and encapsulated ways of interacting with them is a separate concern from navigating and modifying their underlying datastructure, which is an implementation detail.
IMO the "it's just data" benefits of Clojure are that a lot of the common patterns which might have felt like being macro-worthy in CL/Scheme are achievable using things like keywords, which leads to more uniform code (rather than ad-hoc project-specific macros).