I think nuclear energy optimism is largely based on wishful thinking.
Meanwhile nuclear plants go on saving millions of lives by displacing air pollution deaths. China is building high-power district heater reactors to reduce coal deaths in Winter [1].
You're absolutely right that rapid improvement requires high volume. That's probably the fundamental challenge of nuclear right now. MIT is working on shipyard constructed nuclear stuff that has these characteristics for this precise reason [2]. UltraSafe nuclear's microreactors are attempting the same dynamic [3].
Human progress depends wholly on people taking bets like this.
[1] https://www.reuters.com/article/us-china-nuclear-heating/chi...
[2] http://news.mit.edu/2015/new-look-floating-nuclear-power-062...
There are plenty of reasons to be optimistic that the Earth's natural nuclear fuel resources can find a way to be effectively used by humanity. We've done an OK job at it so far. It's relatively safe. It's very low-carbon, and it has been making ~60% of the USA's carbon free energy for a few decades. Not a bad track record.
But it's a bit too expensive. So we just have to figure out ways to make it cheaper.
This certainly doesn't seem insurmountable. I don't think a "can't and won't" curse is at all warranted.