Use the Sidewinder missiles as an example. You're going to run out of munitions very quickly and be left with far poorer capabilities.
Maybe it could be done defensively over territory you controlled, but the 'drone swarm' in this instance really does just sound like a ground-to-air missile.
The smaller your drone the lower its effective range or loitering time. Therefore a transport vehicle such as a jet has to bring the smaller weapon into effective range. The transport vehicle usually also has much better sensors so that it can effectively search for targets. Guided weapons receive additional targeting data from the transport vehicle even after they have been deployed and are heading toward the target.
When people talk about medium sized "kamikaze drones" that are carrying enough fuel to reach a given far off destination they don't realize that these already exist and are called cruise missiles and tomahawk missiles in particular are quite cheap. For the cost of a F-35 the US can buy 100 tomahawk missiles. I still remember that attack where 60 cruise missiles were launched from a single navy vessel at a military airport in Syria with essentially no risk of counterattack.
Most drones are cheap civilian toys. Military drones have had much greater endurance for decades.
> Therefore a transport vehicle such as a jet has to bring the smaller weapon into effective range.
If you are using short-endurancd drones for some reason (e.g., missiles), the carrier can just be a long-endurance drone.
> When people talk about medium sized "kamikaze drones" that are carrying enough fuel to reach a given far off destination they don't realize that these already exist and are called cruise missiles and tomahawk missiles in particular are quite cheap.
That they already exist and are cheap and effective is proof of the point, not an argument against it.
For a drone (even a swarm of them) to compete with a modern jet fighter, it'll need a reasonably powerful engine and sufficient fuel to keep it up in the air long enough to: seek out and engage the enemy, carry a decent payload, and reach a suitable altitude. That all sounds like a lot of fuel and engine power to me, especially if you're expecting this drone (swarm) to operate outside of the immediate vicinity of the launching area.
I see more potential for drones to act as 'screen' for jet fighters once stealth technology is made irrelevant by advances in radar. I foresee the use of, low cost vehicles that resemble fighters (in the ways that count), but carry minimal ordinance (if any) to keep the costs down. They could also provide auxiliary functionality like electronic warfare and scouting.
I mean you could in theory develop a drone that exploded on impact and operated within a limited area, but you've just re-invented the guided missile, with a little extra smarts.