Everyone expected him to drop out earlier, but I think the reason he stayed in as long as he did was to take all the bullets for Kamala until he felt confident all the other potential challengers would get behind her.
What he didn't want to do was drop out early, let Kamala get battered and bloodied by her own party nomination process, win the nomination, then lose to Trump just because she'd been dragged through the mud in the media for months.
The more simple explanation, in my opinion, is that Biden is exactly what some had thought for over a year now - a person getting older and starting to run into cognitive and physical decline. I've watched multiple family members go through this, and the picture of a person in the middle of decline lying to themselves about their new reality and stubbornly refusing to accept it fits perfectly well.