I will say that I drink a smoothie as a (post workout) meal 1-2x per week and the ingredient that really alters that composition is almond butter (and quite a bit of it - 1/3 cup or more).
Nut butters, yogurt and oats should be primary ingredients and fruits should be sparingly added just for a bit of flavor (I throw in 8-10 blueberries which is plenty).
Non-starchy non-sugary vegetables should be primary ingredients if you're not looking for a calorie bomb, though it sounds like you are. Let's say you're feeling generous and your almond butter edges up to half a cup. That's over 700 calories before any of your other ingredients.
It's not healthy to eat whole fruit (like bananas and apples) from the local grocery store?
source?
It's also perfectly healthy to eat a ripe banana if you are doing a physical activity, as the insulin spike will likely help your muscles use the sugar and you won't store it.
With the exception that eating high GI carbs/sugar before doing a high intensity activity is not problematic, because it will be consumed directly by the muscles as "fuel" and might even improve performance. But yes otherwise (ie the majority of the time) I agree.
When you eat food, it takes your body time to digest it all, so you get a slow and steady blood sugar flow. However, when you eat smoothies, all of that food is already nearly ready to be absorbed, so your blood sugar spikes and it's a tremendous burden on your liver to handle all of that food value all at once. Needless to say, smoothies are really not for diabetics or athletes! This is the number one issue with smoothies. A slow and steady blood sugar / nutrition delivery is better for everything.
Outside of that is the sugar issue. Sugar and sweeteners are in everything and people naturally gravitate towards recipes with sugar in it, whether they realize it or not; maybe you aren't using fruit juice, but now you're sweetening with a ton of bananas, or now carrots, etc. We're just helplessly addicted to it.
A third, less known issue is that fruit now adays has been so domesticated that it really has very little nutritional value (esp compared with its wild cousins). It's right up there with milk on "why are we telling people to eat this" type of FDA questions. If you google "healthy smoothie", all you will see are yummy fruit-based smoothies which really are hardly healthy.
If you are after fiber, there are much better things to eat. So in general, fruit smoothies are just not healthy at all no matter what. The really gross-tasting cross-fit type smoothies full of wheatgrass, kale, and other vegetables are much better for you, but who's going to subject themselves to that kind of punishment; might as well eat a salad, at least there's some fat from the dressing. Regarding fiber specifically, flaxseed and chiaseed can be mixed into just about anything with very little taste effect. Beans and lentils are high in fiber.
Honestly, even a no-sugar fiber drink mix would probably be better for your body as a whole than a smoothie.