What you are referring to is the viral load. How much does it take to make you really ill. It matters because your body can fight a few dozen virus cells more readily that it can a few thousand. Sure it can make more antibodies but they take time, time the virus has to replicate, so now you need more antibodies, and yiu aren't exactly sure which ones work.

https://www.bmj.com/content/371/bmj.m4763