Any body holding significant power is at some level accountable to the public, regardless of laws or regulations or how little the public may understand. If the Fed can’t make the case to most citizens that it should exist and retain its power, then it is doomed, and no philosophical arguments could change that.