The specific complaint is that Brutalism is utterly dependent on reinforced concrete. The outer faces are largely cladding and support for the rebar which makes the entire structure viable; concrete alone has terrible tensile strength, and so can't possibly support high-tension structures like Boston City Hall.
That's not just a mechanical observation; it's a major source of criticism for Brutalist works. Rebar ages terribly. Its lifespan is perhaps 10-20% that of pure concrete, and the Brutalist style actively shortens that range. The lack of paint and cladding enables the moisture damage that destroys rebar; concrete is not actually impermeable. (It also tends to create unsightly stains due to rust.) Brutalist buildings need repair quickly, and tend to fail in particularly unpleasant ways - repair is expensive and urgent once decay starts.
I agree that extensions and elevated structures can add value. I'm told Boston City Hall's overhangs and plaza produced a welcoming sitting area for many years. But at the same time, it's currently a foreboding stretch of wasted space in the heart of the city. I don't know how much of that is a changing relationship with political institutions (City Hall is less 'local' and more 'political' today) and how much is the changing role of public space, but it does leave a lot of Brutalist works failing at their original goals.
Absolutely, reinforced concrete is not a recommended structural design tool in modern thinking, but some of that is learning lessons from mistakes Brutalists made, so it's hard to fault some of the over-reliance on reinforced concrete.
Just because a lot of Brutalist works seem to be failing at their original goals with at least a couple decades of hindsight, doesn't mean it wasn't worth exploring those ideals and goals. There's a lot Brutalism has taught us/can teach us about building materials, and the public's ever-changing relationship with civic spaces, and even what the "usefulness" of "ugly" buildings can sometimes be.
Brutalism (and especially neo-Brutalism) didn't allow facades. While that concrete may have rebar, the entire Brutalism philosophy was about not hiding the materials of construction. If a thing was to be made of concrete, it was going to look like it (instead of a faux brick or tile facade or whatever else), and a thing wasn't to be made of concrete if it didn't need to be (at the time that was often the cheapest material for the work).
The concrete extensions out from Brutalist buildings were designed with a function: to create outdoor courtyards and gathering spaces, interesting shades and "cubbies" for socialization. A lot of those spaces don't get used for that much anymore, but that's as much of change in people's relationship with the outdoors as modern a/c has changed the landscape.
There were a couple neo-Brutalist buildings rather central to my University's campus, which lead to my interest in the style, and it was interesting realizing how intentional they were meant to shape the space around themselves. That was a case where you would see a lot of those strange concrete overhangs used and subconsciously appreciated by students throughout the day. I don't think a lot of students realized those spaces were intentionally designed for just that usage.