It is. If you can't act, then it's not your responsibility, but if you can and choose not to, then it is.
In many countries, this is codified in the law: if you see someone drown or otherwise in lethal trouble, you're expected to do what you can to save them. You can't choose not to interact in order to avoid responsibility; if you're aware of the situation and in a position to act, you carry some responsibility.
It is only the case if you don't kill someone to save someone else.
Let's say there will be an investigation because of criminal liability by inaction (civil law jurisdiction). First you aren't in guarantor status (one example is the surgeon during a surgery), and second if you explain that you tought it through and showed that the what-if case is dangerous, you won't be punished for inaction.
So, no, it does not really matter that you are involved by being there.
And now I am sure that many people could develop post traumatic stress disorder because they feel they have failed to save people. I am sure that these people can be helped if they are shown the possible outcomes of action and inaction, especially that action is often more problematic than inaction.
(if you don't click on the link though, you're fine)