pyarrow for example, if you don't have a wheel then good luck building one without using conda. It's not impossible but the developers don't really support it.

Official dependency and build tooling is not properly geared for C extensions. You'll be looking at compiler errors to figure out what dependencies you are missing.