I think it's clear in context that the fig tree is symbolizing the kingdom of Judah, who's leaders Jesus is very critical of. In the gospels, he says they will be disowned and their kingdom given over to another. The fig tree is a symbol of the kingdom of Judah and it really works in context.