Quite right, and penetrating this "I dunno" defense is part of building a criminal case against the high-level traffickers. It's the job of the District Attorney (local), the Attorney General (state), and their teams of investigators; that is, the detectives. To do this they can surveil, pressure arrestees for lower-level crimes, penetrate organizations with undercover agents, and even tap phones and computers—pretty much anything you've seen in an American T.V. show, which is often dramatized but not always as far from the truth as you'd think.

> You could increase punishment for folk buying the goods

Do you mean the end-purchasers? Conceivably you could get some of the shopkeepers, if you can prove they knew the goods were suspect. But there's nothing criminal about buying household goods at a corner store—all the crime happened on the way there.

I guess this particular tactic isn’t an option if you don’t prosecute low level crimes like shoplifting.

For buying goods from random people instead of a distributor? No of course not. Any reasonable person would be suspicious.

[a]: They are a person you don’t know, but it’s not like a flea market where you can’t read reviews before handing over the money

Yes, if one has bought many shirts, bags, laptops from these people; even in this case, prosecutors don't go after the fencers(middlemen) directly. They bust the small guys, who flip for plea deals. Then these 'thieves' wear the wire and sell stuff to these fencers; this is how prosecutors collect solid evidence to convict fencers.

But for criminal prosecution the fault should be proven beyond any reasonable doubt. IANAL, but proving beyond any reasonable doubt a person knew the item he's buying was stolen is more difficult than not.

It's not an unusual concept in common law. It's like manslaughter - you can have had absolutely not intention whatsoever to kill or harm someone, but still be held responsible for their death because you weren't careful enough.