Note that Git commit graph isn't really a tree, but a directed acyclic graph, in which nodes (commits) points to actual Merkle trees (trees/blobs).

(Also, git keeps the full data, not only diffs)