The idea that regulatory agencies act solely on the basis of research and law is not believable, because you see regulatory agencies directly reversing rulemakings that were enacted only a few years earlier (not coincidentally, during the previous administration). It beggars belief that objective research and non-partisan legal theory has changed so much in just a few years (again: spanning a presidential election) that regulations have to be revisited and reversed by new rulemakings.
Look at it this way: if FCC rulemaking doesn't take public opinion into account, why did supporters of this rulemaking feel the need to stuff the comment box with a bunch of fake, identical, supportive comments? They should have had no reason to do that, since the rulemaking was in their favor and (supposedly) being done for an objectively right reason. But the truth is, they feared the politics of it and wanted to create a counter-story that blunted the appearance of how unpopular this rulemaking was.
Again: if that wasn't true, then why did people who supported the FCC rulemaking bother to pump in a ton of fake supportive comments? Why did people at the FCC try to cover up that that happened?
It absolutely is. You’re designing a complex and highly coupled system within a framework of rules and constraints. In the case of regulation, the rules and constraints come from economics, while in the case of operating systems they come from computer science. In both situations, there are no known optimal solutions—nobody has designed a mathematically optimal way regulate electricity prices, much less structure an electricity market, just like nobody has designed a mathematically optimal way to manage memory, much less structure a kernel. Thus, a lot of judgment and heuristics and trade offs are involved at every level, and the exercise of that judgment is going to depend heavily on personal philosophies. Margaret Thatcher approaches regulation very differently than Franklin Roosevelt, just as Linus Torvalds approaches operating system kernels very differently than Andrew Tanenbaum. If you hire Tanenbaum versus Torvalds to design you an OS kernel, you can expect a very different result, based on their expert judgment.
Unfortunately, a lot of people think of regulation as just picking from a menu of policy choices, and fail to realize that what you’re really doing is designing economic systems and markets.
A lot of these people vote and donate to candidates! Some even get elected to office. Therefore you cannot dismiss their impact on the regulatory process, even if you would like to.
Because it is ultimately the political process that determines which people--which personal philosophies--will have the resources and power to make regulations.
Phrased another way: if the experts say "doing A will work but have tradeoff B, while doing C will work but have tradeoff D", it's useful to know whether the country prefers B or D so that the commissioner can pick A or C accordingly.
The independence of agencies like the FCC is meant to be independence from any individual President, so that the commissioners are less responsive to the whims of the politicians than the regular executive departments. This is why there are commissioners from both major parties, why their terms of office are longer than a presidential term of office, and why they can only be removed from office for cause rather than merely for making a decision the President didn't like.
Your argument that they should defer to the wishes of the executive is exactly counter to the structure and purpose of how they're set up (even if I admit they're less independent in practice than in theory).
As a point of comparison, regular executive departments are headed by a single political appointee of the president's party, who customarily offers to resign when a new party enters the White House, and who can be fired with no notice for any reason or no reason.
I don’t see how that’s a reasoned basis for picking one versus the other. If a court did that, would you say that was a reasoned decision? (“We have these two competing interpretations of the 14th amendment. The majority of the public opposes interracial marriage, so we will pick the interpretation that allows states to ban it.”)
As to your other point, independent agencies are set up so that the President can’t micromanage specific policies. But the heads are nominated by the president precisely so the executive can set the overall regulatory agenda and philosophies. Remember the context in which the APA was created. The government regulated things like what routes freight trucks could run and the prices. You didn’t want direct executive control over those things. But broad agendas, such as ensuring more freight service to the Midwest? That was squarely within the administration’s power to influence the agency.
The 14th Amendment example is about statutory/constitutional interpretation, whereas the type of ruling we're discussing here is not.
Agency regulations (and common law rulings by courts) often make policy judgments among multiple valid possibilities. Sometimes public attitudes appropriately affect the policy judgment, though of course not always.
Of course agencies and courts don't make popular opinion the primary driver of their rulings, even when they consider it. And I'm certainly not suggesting they should.
As for why the APA was created, that's a very complex question, but that seems more about systematizing the regulatory rulemaking process as delegated from Congress than about the difference between an independent agency and an executive department (both of which are governed by the APA). We're talking about the kind here that's supposedly independent.
After all, they serve the public despite the indirect accountability, and often there is more than one path which the experts deem viable. Public preference can be an appropriate in choosing among the viable paths.
In this situation, when the coordinated fake comments line up with the governing politicians' views, and when both run contrary to the weight of real comments from experts and non-experts alike, it casts doubt on the genuineness of the FCC's good faith in considering even the expert public comments, and therefore on the validity of the rulemaking overall.