> Saying "Children aren't born smart" is equivalent to saying "Intelligence is determined entirely by one's environment", if I understand the claim correctly.

You don't understand the claim correctly. The claim is that environmental factors (particularly being talked to) are essential to development of intelligence, not that environmental factors determine intelligence entirely.

The distinction between necessary conditions and necessary and sufficient conditions is pretty critical.