Removing Chevron deference moves authority from the administrative agencies to the courts, who are not only also unelected bureaucrats, they are more insulated from democratic forces since they have lifetime appointments. Agency heads are political appointments and change with every president.

They're judges, not bureaucrats; they cannot issue regulations governing people's behavior; and you entirely ignored "unreviewable."