You said "we’ve let in ~5 million illegal immigrants" but this is the count of 'actions', some of which are almost certainly with the same people multiple times. This number also includes some people who weren't even trying to get in illegally:

"Inadmissibles refers to individuals encountered at ports of entry who are seeking lawful admission into the United States but are determined to be inadmissible, individuals presenting themselves to seek humanitarian protection under our laws, and individuals who withdraw an application for admission and return to their countries of origin within a short timeframe."

It's not totally clear, but it seems that it also includes other "border enforcement" actions that are not necessarily related to immigration at all, e.g. intellectual property and drug seizures.

So it has nothing to do with what you were claiming.