I'd guess the singularity is decades away not centuries.
Genetic engineering babies for higher intelligence is already here. But I'd guess it'll become more effective and common place in decades.
The intelligent tend to clump and interbreed. They breed less, so % might go down but numbers are probably relatively stable.
Add to this many counties like the billions in China currently don't have this problem atm. When they get their education system / poverty sorted they'll add huge numbers to last us out the centuries.
Plus I also think communal intelligence should not be under rated. Even if g drops, systems of group collaboration might keep the issue at bay.
It's also quite possible that many worlds are far more prone to catastrophes than Earth. Which has had a few hard knocks of its own -- asteroid bombardment, toxic atmosphere, snowball Earth, hothouse Earth, supervolcano eruptions, climate change, more asteroid impacts, etc.
On a world where the system kept getting reset every few million years or so, it might be hard for advanced life, let alone civilization, to get off the ground.