The fact that the head of the FBI is an appointee of an elected official, there is some public accountability through voters if there is enough transparency to inform voters of what is going on.
I suppose the same is true for most local police though. As it stands, we don’t get much transparency. For a democracy to work properly, the public needs to be informed. The stonewalling by law enforcement at all levels prevents the public from holding them accountable.
You cite transparency, which is one aspect- but the other interesting problem is the incestuous relationship that our intel agencies have with the media as well.
How much political news coverage was shaped by "anonymous sources" from these intel agencies?
I forget who it was- but I remember one retired agent speaking pretty casually about how the FBI leaks information to journalists by sharing information with retired agents that still hold their clearances.
Accountability is further complicated by the fact that the entity is political in nature, so the FBI has actual partisan backing through politicians, the court system, and even down to individual jurists.
I honestly don't know what the solution is myself.