>> a lot of fuel just getting through the thick low part of the atmosphere

That statement is misunderstood. Google "rocket ISP". Rocket engines work less efficiently at lower altitudes, and cannot be properly optimized due to the rapidly changing pressures between sea level and the near-vacuum at altitude. They burn extra fuel due to this inefficiency. And since rockets are at their heaviest in the lower atmosphere, even little inefficiencies really add up. Atmospheric drag is next to nothing in the equation. Even speed isn't the issue. It is about how they cannot design an efficient engine bell to cover the initial climb.

I've also heard that fuel is not the main cost contributor to a rocket launch. In fact, it is a really insignificant portion of the cost. It is the vessel that contains giant amounts of fuel that is costly. Hence reusability from SpaceX makes sense and so does launching the rocket from high altitude to avoid building the first stage all together.

But it's not just altitude, right? Acceleration and Speed are important factors to getting to orbit (not that I'm a rocket surgeon).

Right. The closer to sea level you are, the denser the atmosphere you're pushing out of the way as you accelerate - thus, the more force you need to use in doing so, and the lower the acceleration you obtain from the propellant you expend.

So if you can get above the thickest part of the atmosphere, say attached to a giant goofy-looking airplane, before you light off your engine - rather than having to push your way through that same thick air on your own - you need less propellant, overall, to reach orbit, which means you can devote a larger fraction of your launch vehicle's mass to payload.

Rockets are not moving very quickly in the lower atmosphere, which makes a real difference.

Space shuttle was 3 g - 1 from gravity = 2g net acceleration, but that's kept low for passengers. Even then 2 g net ~= + 44 MPH every second.

So, first 15 seconds your going under 660 MPH which is not that fast, but you get ~1.4 miles up. Because drag increases with speed very quickly the next bit is harder, but you very quickly get above what an aircraft could take you with ~5.6 miles at 30 seconds. Note: Higher g's mean more drag in denser atmosphere, but less gravity drag it's a meaningful trade-off.

However, aircraft's velocity is very useful essentially saving those first 15 seconds of full burn, but comes at the price of needing more structural elements to support the hanging rocket.

Drag at low altitudes is much worse, so perhaps just bringing it to a higher altitude is that much more efficient?

Build a 3 mile tall launch platform that is also an EM rail. Or better yet, BORE a tunnel through a really tall mountain.

https://en.wikipedia.org/wiki/Pico_de_Orizaba