This is worth treating carefully. There are sources like Ibn Kathir and al-Tabari writing centuries later which claimed that Zakat collectors would return with unspent funds because eligible recipients could not be found. Even if we take this at face value, that's a leap from there being no poor people left. Did it apply to the entire empire or only a few regions? How likely was the collector to actually find recipients if they existed in a given region? etc...