But you wouldn't tell your kid: we are going to visit your paternal grandmother, right?

That is the distinction I tried to make above between academic and everyday language.