When backpacking with kids, we only relocate every week or two, not every few days. We have rest days for schoolwork. When cycle touring, we don't go more than 50km in a day. We eat a lot more icecream. But it is still unplanned, unscheduled and intimate.
There is no way I'd do it again with the kids in tow. Sure, it goes well 95% of the time. That 5% time when you can't find anywhere to sleep except a trailer park from hell or a disgustingly dirty backpackers ran by drunk/high perverts is something I'd never risk with young kids.
One of our best vacations was one of the few package tours we did, this time to Egypt. It was via a company who do backpacker-style vacations. It was fantastic because our guide was excellent and we had contact with the locals. They used public transport wherever it was possible too, which although uncomfortable it felt better. We did a package in this case because it's also a country which simply isn't safe to be driving around in as westerners..
I was lucky enough to do a trip a while back with a relative that gets travel discounts and comps, and we combined budget backpacking with stays at some of the finest hotels in Thailand, including this place: https://www.sixsenses.com/resorts/yao-noi/destination
I recall that the breakfasts were sublime - they had something like 20 different varieties of ice cream, and fantastic pastries. The chef was originally from California and would talk you through the various delights on offer that morning. A fantastically peaceful, beautiful place, and an experience that's difficult to get on a budget.
For example I will never ever do any other type of vacation, it doesn't matter if with kids or being old (of course the nature of the trip will reflect those aspects). I will never book or plan whole vacation in advance, rather just have flight tickets, info about possibilities, and adjust plans as trip unfolds.
One is getting much much more bang for your bucks this way, especially since you generally spend less bucks to start with. Intensity of travel is much higher, one interacts much more with locals, tastes true local foods etc. Its one of those things that the more money you put in, the less rewarding and more bland experience you are getting back.