The most common attempt at a solution to this problem has been "civilian oversight," which typically takes the form of a new department of the city government that is responsible for auditing the police department and/or investigating department actions (in response to citizen complaints or on other triggers). The civilian oversight department is typically firewalled from the police department through means like a prohibition on staff moving from one to the other and an independent governance structure.
Because of collective bargaining agreement constraints there's usually still a PS division and some confusing work breakdown, where one audits the others work or the two perform independent parallel investigations or similar. There are some good reasons for that (doesn't necessarily make sense to have an independent civilian oversight agency reviewing every minor policy violation), but it often feels like waste.