Having the codebase be as small as possible is a laudable goal - with browsers it is difficult since Web standards are fairly complex on their own, and being cross-platform brings along a lot of weirdness.
An important aspect of security is compartmentalization - for instance, using separate sandboxed processes for web content vs. the main UI (which runs with full user privileges).
Sandboxing is a good example here since it improves perf/stability/security but also adds to the size and complexity of the code.
Only in an abstract "we might have been able to implement security/performance improvements faster if we wouldn't have had to worry about breaking addons" sense. Any other effects are restricted to those people actually using the addon and don't affect everyone else.
A powerful extension interface can also be used to improve performance/stability: while e.g. adblocking certainly doesn't come for free, its cost should be more than offset by not running all that crappy code pulled from ad networks.
That's what extensions are for. Those who want it, can have it, those who don't, they won't. Everybody wins, no need to raise emotions.
Firefox is open-source and (fairly) easy to build - if you really want the utmost control then I'd suggest this is the best way to do it. You can easily do a build that has no Pocket integration.
There's a cost to the level of configuration that I assume you're advocating here. Not saying we can't get better here, but there's definitely an impact on code complexity, QA test combinations, and stability.