I would argue you absolutely need to be testing the internal details. That is the entire point of measuring branch coverage and performing mutation testing. Unit tests are not black box tests. They need to know that for special values, the unit has to follow a different code path but still produce sensible output. Reading the documentation of a function is not sufficient to determine what edge cases the unit has, but testing those edge cases is often critical to verifying that the unit adheres to its specified behavior under all conditions.

As for the smell, sometimes things are irreducibly complex. Some things in this world do require tedious book keeping. All the refactoring in the world cannot change the degrees of freedom of some problems.

Tests on consumers should not test branches of subordinate units. If you did this then the number of tests would explode exponentially with the number of branch conditions to handle all the corner cases. If a private unit produces a list of objects, but has special cases for some values of its argument, test those branches to verify it always produces the correct list. Then just make sure each caller does the correct thing with the list of objects. That is the purpose of separation of concerns: the consumer does not need to know that some values were special.