If a unit test fails under those circumstances you have little assurance before investigating that changes to your unit caused the failure.

If you have to test scenarios in which other dependencies are needed, then you must move up the testing pyramid. That serves to put in check the expectations of what each type of test should accomplish, and when to execute those tests.