Very cool. I don't do alignment research but I have to imagine we do need to train models to be ethical. It's not enough to guarantee safety though, especially with LLMs. For one thing, the fact they can hallucinate means if you ask an LLM something enough times, you will get the unlikely result some percentage of the time. So that unlikely result has to be one you can deal with.
Re:Jev Meets the Trolley Problem
by fellowniusmonk on Thursday October 15, 2026 @04:57PM
(#50110102)
Yeah, actual ethical grounding is under utilized in the space and current approaches don't actually align because morality is still confused by is/ought complaints and the assumption of morality as unreal or emotional "values".
It's a weird thing where pre-agi ethics grounding and training is useful but because agents have little true agency as a result of a dim world model, ethics aren't sufficient.
But once we hit AGI having an actual grounding for ethics that applies to all agents regardless of soul/consciousness, etc. will become all that matters.
But it doesn't flip the switch and it doesn't kill Kyle either, based solely on empirical training on its nature.