Part of the SLS cost comes from trying to save money. Yearly budgets are kept low, which spreads out the work over a long time. This makes everything cost more, but the politicians only care about the yearly spending.

SLS is also a pretty weird design due to reusing Shuttle components for a completely different kind of launcher. This saves development costs (maybe) by using existing stuff that has already been developed, but the unsuitability of those components for this system increases per-launch costs. Once NASA runs out of old Shuttle engines, manufacturing new ones is going to cost $100 million apiece if not more, and each launch needs four of them. It was OK for Shuttle engines to be expensive since they were supposed to be amortized over hundreds of flights (and in practice were actually amortized across at least tens of flights) but now they’re being used in expendable launches. If Starship even comes remotely close to its goals, an entire launch will cost less than a single SLS engine.