Point. Edited and hopefully clearer. I try to keep non-first language readers in mind, but vernacular crept in before a second cup of coffee.

IMHO, "this isn't to say" is a common expression in English, so I didn't register it as a negative. But you're absolutely correct.