Test primarily against external APIs. Those should be stable against change anyways, so that the external consumers can rely on it.

Generally don't test against internals much, except if an area is shown to be buggy and hard to exercise externally.

Prefer (well tested) libraries for generic functions/algorithms. If you have to implement one, put tests on it as if it was external API. Consider publishing it independently to make it external. If needs change, just drop the dependency.