This is called immunotherapy and it is not mutually exclusive, and it is being worked on to treat certain diseases. However it's not nearly as simple as you might think and many immunomodulatory drugs have considerably worse side effects.

Immunomodulatory drugs to some extent "predate" modern antibiotics before the antibiotic boom and the advancement in modern wide spectrum antibiotics immunomodulatory drugs were commonly used to treat severe infections.

The side effects of these drugs are very very severe, can be fatal, and can cause hereditary malformations Thalidomide is probably the most famous case of a "wonder drug" going very very wrong it is still used today in extreme cases of leprosy and to treat some cancers.

https://en.wikipedia.org/wiki/Thalidomide

But just to be clear these type of treatments are just as "designer" molecules as any other drug, there are no magic fixes in medicine you find the most effective vectors to attack a disease and you use them.

But overall >Crazy - I never actually took a step back and wondered if we could boost our defense as a strategy.

This is a bit silly, we have had vaccines before we had antibiotics :) we wash our hands, keep ourselves clean, dress wounds etc. these are all "defense boosting" strategies, and they both predate and coexist with "offensive" treatments.