What I don't know is if the cost of a small road per tonne-mile service life is any different from the cost of a large road per tonne-mile service life. If the cost difference is negligible then I don't see the problem.
Maybe it is that different parts of the government fund the small roads than the large roads, and the parts of the government that funds the small roads doesn't have the budget to do much road work?
Do heavy vehicles generally use these small streets, it seems like it wouldn't be worth it in a heavy vehicle with most small streets.
The cost differences are huge. A primary road will have been constructed to tolerate the weight and frequency. It's much more expensive to maintain such a road.
In some cases, yes. Maintaining and improving roads will be paid out of different budgets. Federal, State, (sometimes) County, Local Municipality. Tertiary roads are almost certainly out of the local budget.
If it's not expressly prohibited, there's bound to be heavier traffic on these tertiary streets. The heavier vehicles do far more damage.
(There's also safety to be concerned with.)
It's okay to route traffic through these areas when it's an emergency. Occasional use is not going to be a significant impactor. However, with regular use, it's going to result in rapid deterioration.
If you really want to dig into this, here's a link to a study:
https://www.lrrb.org/pdf/201432.pdf
There are many, many more. If you're curious about the weight issue, it's known as ESAL and this page is pretty easy to digest:
Additionally, there is the safety aspect. It is more than just pedestrians at risk, there are other automobiles and structures to be concerned about. Tertiary streets, for example, probably won't have things like guardrails.
Areas zones as residential should probably not have much through traffic. It is expensive and dangerous. The roads constructed to deal with that traffic are also built to withstand that higher load. Routing the traffic elsewhere will not significantly reduce the maintaining of said roads.
Err... I modeled traffic. Hopefully that explains it. There's some good write-ups as complaints about Waze, if you want a keyword to find more info.