The whole point is that the technology itself and its capabilities are not well defined, people are constantly inventing new applications and new methods now at breakneck speed, so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.
Before 2016 or so, "AI Safety" meant the types of things I listed above: how to design AI systems that are safe to use in safety-critical settings.
Unfortunately, the term "AI Safety" has been overrun with effective altruist longtermism folks talking about something closer to https://en.wikipedia.org/wiki/Existential_risk_from_artifici... which is ill-defined and, frankly, usually completely vapid nonsense.
> so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.
No, no, no!!! AI safety methods -- methods that really increase confidence in real systems -- are not generic. ALL of the useful AI safety work I know of is deeply related to one or more of: the specific model architecture, the specific optimization algorithm, the specific data, or the specific application. Almost always all of the above.
Pressing the idea that AI is dangerous makes it seem like these companies are even more powerful than they are and could drive up their stock price. When the AI Safety people get into some conflict and get fired they are really doing their job because now it looks like big tech is in a conspiracy to cover up how dangerous their technology is.
Using ML for object detection, object tracking, or prediction on an L2-L5 driver assistant system? AI safety research sounds like a capability you'd really want.
Using ML for object detection, object tracking, or prediction on an industrial robot that is going to work alongside humans or could cost $$$ when it fails? AI safety research sounds like a capability you'd really want.
Using classifiers or any form of optimization for algorithmic trading? AI safety research sounds like a capability you'd really want.
Building decision support systems to optimize resource allocation (in an emergency, in a data center, in a portfolio, ...)? AI safety research sounds like a capability you'd really want.
Hell, want to use an LLM as part of a customer service chatbot? You probably don't want it to be hurling racial slurs at your customers. AI safety research sounds like a capability you'd really want.
Unfortunately, now "AI Safety" no longer means "building real world ML systems for real world problems with bounds on their behavior" and instead means... idk, something really stupid EA longtermism nonsense.