The high-adventure trips were the best part for me. The multi-week canoeing trip in the boundary waters between Minnesota and Canada (northern-tier), multi-week sailing trip and snorkeling in the Florida Keys, the Philmont backing trip, etc. Our troop took backpacking trips on the Appalachian Trail in Georgia, Talladega national forest in Alabama, etc. Many fun memories with family and friends.
I was in 3 troops as a kid due to moving around. The troop in the Chicago area was a fantastic example of a by-the-book Boy Scout troop. The scouts ran the show and the adults only got involved if we were about to do something very stupid (making non-dangerous mistakes and learning from them is an important part of maturing), or if the decision required an adult to sign off (spending troop money, etc.). The one in Alabama was more laid back, kind of boy and adult led, but the adults definitely did more of the work than the chicago area troop. The one in the bay area I joined ended up being a rich kid dick measuring contest and I quickly left the troop, which was the end of my time in the Boy Scouts.
That's the single biggest problem with scouting - your experience is going to be largely dependent on the adult leadership. You can have dramatically different experiences depending on what pack or troop you're in and leadership changes often make what was a good experience a poor one.