It's not clear to me what exactly you expect to Sync.

Container settings (names, colors, icons, assignments) should be synced (if not, please file a bug). Open tabs will be synced, not sure how they will open on another machine (considering URL-to-Container assignments should be synced, they should open in the respective containers – maybe there's a possible improvement here).

I expect the different Cookie stores to be synced as well, if not, that's a bug.

I can't even get them to persist after restarting FF. :/

Please describe the behaviour you're seeing vs. expecting. Is the session not restoring the URLs in the right container? That sounds like a bug to me. The container names (if you change from default) should be stored in the settings – if not, something is wrong on your machine. If you want to always open some specific URLs on start, pin those tabs.

I updated the container names, colors, and icons. Opened a few tabs here and there. Installed a user agent switching plugin. Restarted to enable said plugin. While the session was restored, including the containers, the non-default container names/meta info were not. Tabs in non-default containers showed up with no labels, and there was no apparent way to open up a new tab in one of the custom containers.

Also, changing a container's color does not change the color shown on existing tabs.

Please check and file these bugs here: https://github.com/mozilla/multi-account-containers – I assume you are using this extension?