I'm the exact opposite. I start with the highest level integration tests to test the API out. I iterate on this process quite a bit until the API is more or less settled down, and then I start implementing the API with appropriate unit tests.

As much as I'd love a very cleanly implemented API, in the end many more devs will be using the API, so the use of it has priority over an API that is easier to implement.