They're definitely trying that as a loophole, but it won't hold up. "No action/response" can't count as consent, and so if you never granted it, then they don't yet have it to begin with and aren't allowed to collect.