Nope. I’m not mistaken. What you’re talking about only applies to Xbox 360. For Xbox One, you lose access to your claimed Free Games with Gold when your subscription lapses until you resubscribe. The same is true for the free games to PlayStation Plus subscribers.
Re:When will Google shut down Stadia?
by WorldMaker on Thursday November 21, 2019 @01:51PM
(#21597860)
Yes on the 360 because its Store was never built with Memberships in mind. No on the Xbox One (and PC where applicable) where a lapse in your Gold status will make those games unplayable until you rejoin.
Games With Gold are actually free. You need the Gold membership to get them, but you can unsubscribe - they're yours.
Edit: You may be thinking of Game Pass, which behaves in the way you describe.