Most of us, exact sciences pilgrims (speaking as if my degree makes me worthy of being included in such a group of individuals), should try to go outside more. No offense, but our biases are too great.
We should stop repeating the marketing speech. It's getting tiresome.
Right now the existential risk is fully in hands of humans, unless someone decides to free one poorly trained agent into the open Internet, then you'd have more problems. Stuxnet level stuff.
Did you saw that thing Whisper? What if someone trains something like that, but fine tuned for automatic-hacking? and automatic retrieval of new exploits, and automatic re-shuffle of code to hide from security software, and pum, the thing is flying solo into the wide Internet, just f*cking aways stuff and hiding from silly humans. No conciousness, not chatGPT-like level of intelligence, but 10x the danger level of older non-intelligent worms.