I always figured it was a combination of the volcanic winter of 536[1] and the Justinianic plague[2] (which happened right as the eastern roman empire was reconquering the western empire to reunify).

[1]: https://en.wikipedia.org/wiki/Volcanic_winter_of_536 and https://www.science.org/content/article/why-536-was-worst-ye...

[2]: https://en.wikipedia.org/wiki/Plague_of_Justinian