Nonsense, if I owned the whole building I would say "my [more familiar term than 'building']".

Perhaps an example will put you at ease:

"Oh could you help me take these to my flat? I know they're heavy, but don't worry, my building has a lift!"

Substitute apartment and elevator if so inclined.