Been a while since I read both books, but other than a vaguely similar shape, what makes the Affront clearly a juvenile version of the dwellers? I read the wiki you mentioned and there is a story suggesting an early version of contact rescued them from dweller hunts, but no mention of where this idea came from. I'm a huge fan and believe I have read everything Banks has written and do not recall the story.

This is probably right and very saddening. Oh well, I guess we can still assume that Against a Dark Background is set in the Culture universe, just very far away.

Or it's a simulated world! After all it has things like the Lazy Gun.

Just checked out the wiki myself. That reads like fanfiction.