I don't think thats true, modeling the data as vectorizations and having paths between them is entirely different from say, building a decision tree. The attention head is also something entirely different.

And the model is capable of doing much more than just making simple predictions, it can build generalized abstractions and form coherent outputs on text/logic its never seen.