The AI just isn't there yet if we would even want it to reach that point. You have to have someone in communications range to make kill decisions, confirm target designations, and keep the expensive parts of your swarm intact. Drones are pretty awesome as front line aircraft, but they haven't replaced humanity yet.

Then tens of thousands of people directly controlling those drones. We have the manpower and the ability to build these drones. Azerbaijan and Ukraine basically kicked ass with essentially DJIs and grenades.

You are correct. However, the people need to be close to such drones. DJIs cannot be controlled from very far away. Similar to non-improvised drone munitions like Switchblades. For supersonic aircraft you need something a bit different.