Seriously, the responsibility lies with whomever made—or okayed—-the system. One-off things, especially if they are hard to foresee, might get a partial pass. Otherwise you ought to be carefully testing and evaluating your own system instead of blaming gradient descent.
And generally speaking these are very different people, but that is besides the point. I am still unsure why we are talking about testing and evaluating, as if we are only talking about failure cases.
If I set the alarm clock for 7AM and it starts ringing at 4AM, the fault is with the clock designer. But if I set it for 4AM and it starts ringing at 4AM, it is somewhat silly of me to start blaming the clock designer for my disrupted sleep, isn't it?
And we are not talking alarm clocks, we are talking about systems that make complex decisions based on hugely multidimensional data. Decisions that we provably cannot explain. There is no amount of testing and evaluation that can ensure that these systems are error free. But when they work, they do work well and provide a competitive edge over human analysts.
This is not true in case of machine learning. Yes, there was a person or people who designed the model, gathered training data, trained the model and deployed the result. But the decisions the model is making as part of its function are not made by these people.
Most of the time, people who trained the model cannot even tell you why the model is making the decisions it is making. Tractability of ML models is currently a very hot topic in the field and progress is being made, but a lot of currently deployed models are inherently intractable.