Even virtualenvs themselves can cause the breakage. Take "pylibtiff". It embeds a copy of libtiff. But, it's old and incompatible with other libraries on the base system. This can be libraries it depends upon, or it could be another copy of the same library. Either way, virtualenvs can cause more problems than they solve.
If you want to use multiple versions of Python and pin them, you should use something like asdf [1] and its Python plugin [2]. It's a version manager with support for many languages and more. I used it for Elixir, Node, PHP, Ruby and even PostgreSQL (I need different database versions in different projects.) I never used it for Python because I'm OK with being on the latest version so far.
With asdf I would expect to do something like this
asdf install python 3.7.5
mkdir my375project
cd my375project
asdf local python 3.7.5 # pick the version to use here
python3 -m venv py375 --python=~/.asdf/installs/python/3.7.5/bin/python
~/.virtualenvs/py375/bin/activate
You mean you aren't familiar with Python's virtual environment system exactly intended for isolating development dependencies from system ones but you're blaming the distribution. Please.