It's a little more complex than that. You get branches (which manifest as knots in boards) where there's sunlight that can reach the leaves on the ends of those branches.

Trees grown among other trees have to grow up to reach sunlight. The innermost part of the lower trunk might have knots from branches when the tree was young, but once they reach harvestable size, there's a fair amount of clear lumber down low.

Trees grown in the open don't have the pressure to grow upwards, and end up with branches down low throughout their lives (from the center all the way through to the bark). As a result, there's very little clear lumber in them.

Disclaimer, I'm a furnituremaker, not a forester. The above is somewhat better than pure speculation, but likely misses some important bits.

The core (heartwood) is the worst part of the tree. Now the trees are harvested so young that the entire things are junk heartwood.

The sapwood is an approximately constant radial dimension over the life of the tree once it gets big enough to form heartwood. Sufficiently old trees are more heartwood by percentage than sapwood.

For hardwoods, it's typically the heartwood that's more desirable. Walnut and cherry both have pretty strongly contrasting sapwood that does not match the customer's expectation for the color of those woods, for instance.

That would be my guess for at least part of the reason, too, but it'd only be a guess. I also wouldn't be surprised to find out that tree species plays a role, and that we plant species for harvest that simply produce worse lumber than what used to be widely available (as old growth).