Quoting the relevant part here:
> Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof
So reinterpreting that in modern words: Congress shall not establish a Church of the United States and Congress shall not govern religion. The phrasing comes from Jefferson, and you are right that it is ex post facto the Constitution, but the idea is very Puritan and Jefferson himself was referencing Roger Williams (who established the first Baptist Church in America over 150 years prior) and his idealistic "Wall of Separation"[1]. Even restating the establishment clause with terminology, I can't read it any other way but as a conceptual separation of Church and State. The language does not need to be literal to have this effect.
Your quote from Adams doesn't contradict this either because it reads like an assumption that the Constitution does not need to do more than this to address religion because the American people are already moral and religious, therefore to the extent that it does so, it is adequate, but it would not necessarily work well for a different people.
[1] https://myemail.constantcontact.com/Roger-Williams---ORIGIN-...
The concept of “separation of church and state” is much broader, and comes from the writings of Thomas Jefferson—who admired revolutionary France and its secularism. But it has nothing to do with either the Constitution or the First Amendment. If we are picking random quotes from founding fathers, you could reach the exact opposite conclusion by quoting Adams instead: “Our constitution was made only for a moral and religious people. It is wholly inadequate to the government of any other.”
It so happens that, because the Establishment Clause is a specific limit on the federal government’s ability to interfere with the state established churches that it makes no sense to incorporate the establishment clause into the states via the 14th amendment. (Just as it makes no sense to incorporate the tenth amendment against the states.)
But my point remains even if you’re just talking about the federal government. The Establishment Clause is just not getting at the “separation of church and state” concept.