Yet, while I lack any deep understanding why they would re-emerge, paleontology seems to show a long history of mass extinctions down to very low levels, then re-emergence of more complex forms.

One theory that I read of a few years back posited that life, and higher forms in particular, are more efficient at increasing entropy, i.e., diffusing heat, and so are systemically favored (seen nothing on it in a few years, so no idea of its current state; if anyone recognizes it and can provide a link, thanks in advance!).