If you can pip install those libs and they work on your platform then it looks like this should work--it's just a nice opinionated wrapper around pip, virtual env and other Python tooling. If you need conda installs because those libs have specially compiled native depencies (boatloads of fortran code, etc) then you're going to run into trouble and probably should stick with conda.

Another issue is that PyTorch installation requires the dependency resolver to scrape the dependency host's HTML like a RSS reader because pypi doesn't automatically store metadata.

https://github.com/python-poetry/poetry/issues/6409

Yeah anything that needs specially compiled or optimized binaries is an enormous pain in the python world. You really want conda for those cases.