If I didn't use pointers, I would have 2 errors. One would be about the use of an incomplete data type (Expression), and the other about trying to put something that doesn't have a "trivial constructor" in a union (String, Value, and std::vector<Expression>).
I can't use references either, because I would have to initialise them in the initialisation list of my constructor, which I can't do until I know the tag. I could still initialise the tag first, but that's specified by the order in which the data members are defined, rather than the order of the initialisation list —a rather dangerous idiom to rely on, should anyone modify the code.
You are right about the issue with std::vector and incomplete types. It will be possibly resolved in the next standard. In the meantime you can use something like boost::container::vector which explicitly support incomplete types.