My understanding is that: (a) the three big Abrahamic religions all place the most emphasis on worshiping as a congregation on different days and (b) among each group, a minority of people who identify with the label actually go to worship with a congregation regularly. Would it really be _so_ bad if you had one big fancy palace which switched from hosting Muslims on Friday, Jews on Saturday, Christians on Sunday and could be a museum / community space / charity space Mon-Thurs?
As for an interfaith building, it's an interesting thought. There might be issues with ceremonies such as weddings, funerals, and events. There is the Abrahamic Family House in Abu Dhabi, but it's not a single building. I thought something like this might exist in Jerusalem, but I couldn't find any..
The Visigoth Kingdom was completely wiped out by Muslim invaders.
It didn't happen by some committee decision, but mostly by killing everybody involved with the old version. That's an important detail.