This is the idea behind modern monetary theory (MMT) and in that context taxes are only used to prevent against inflation. Borrowing money doesn't increase inflation because it doesn't add new currency. One problem with MMT is that makes it very easy for governments to spend lots of money and very unpopular for politicians to support raising taxes because we have infinite money. It's why you don't let your kid eat dessert before their vegetables because they wouldn't eat their vegetables.