One is certainly not, unless one is not well read. The g-factor is one of the most stable findings in psychology. It is well established and well defined.

If you're calling g-factor "that which remains after you have eliminated all environmental factors", then you're not using the common definition. See https://en.wikipedia.org/wiki/G_factor_(psychometrics). To challenge your other assertion, I'll quote the article:

> The measured value of this construct depends on the cognitive tasks that are used, and little is known about the underlying causes of the observed correlations.

(We've had a lot of discussions of IQ on Hacker News. My observations suggest that everyone who supports it in more than 3 comments in the same thread is a scientific racist with a poor understanding of the research on IQ.)

Working in psychometrics I’m in the somewhat conservative “g is simply shared variance of many tests measuring human abilities”-camp.

https://www.jstor.org/stable/1412107?origin=crossref&seq=1

I’m not subscribing to the notion that g should be controlled for environment, quite the contrary, but if you do, what is left is the part of g which is genetics.

EDIT: The bit of knowledge I have comes from being published in psychiatric epidemiology on the topic of cognitive impairment and substance use.

That's not the claim you were making earlier. (Although, strictly-speaking, it's still wrong: many factors other than genetics are involved in producing a newborn infant, famously epigenetics. If you classify these all as "genetic factors", then yes, the claim is tautologically true by way of redefining words.)

I'd be interested to see how you'd go about controlling for those other things: so far, I haven't seen anyone manage it.