Right, that's a risk you take doing local::lib against system perl.
I will often use the system perl to bootstrap https://p3rl.org/Perl::Build which makes building -one- perl for local use easier than perlbrew/etc. - and then https://p3rl.org/App::plx for managing per project environments if I need that
Post-upgrade either various modules weren't found (because they were installed in a versioned path), or they didn't compile because the XS components had been compiled against the prior version of Perl.