I think something has been lost in translation here. At least in the US and UK, responding with "Not really" to a statement usually implies that the statement is completely incorrect. As in, you aren't offering a simple correction from "everyone" to "a majority", but rather suggesting that it isn't even a majority.