You're forgetting the cost of jailing them in the first place. Not only is there an upfront cost of $106k per year(assuming we're talking about CA)[1], that person is also not working(or paying taxes) or gaining any useful skills. What they're doing instead is hanging out in a population of other criminals and participating in a social order that incentivizes if not downright mandates behavior that in itself would be considered criminal in wider society. I don't think it's so far fetched to say that the cost society pays to incarcerate someone with the goal of preventing them from committing more crimes can exceed the cost of the crimes themselves. Deterrence is the only possible way to justify these costs(And prisons do have nonzero use as a deterrent), but the current prison situation makes it clear that prison's RoI growth has some seriously diminishing returns that started at a funding level far lower than the current one.

[1]https://lao.ca.gov/policyareas/cj/6_cj_inmatecost

I've absolutely heard this rhetoric about short jail terms before, though, about any number of crimes. Is there data to back this up?

I think the point was that the number of thefts made outside the prison by people who are in the prison is necessarily zero, unless they escaped. Do you need data to be convinced of this?

They're in prison until they aren't and at that point they may steal enough to make up for the stealing the imprisoning prevented.

The point of adding a 10s delay on incorrect password attempts isn’t to make it impossible to brute force passwords, it’s to slow an attacker from 600 tries per minute down to 6/min. This is enough of a deterrence to make brute forcing impractical.

If you have to wait 7 days between train robberies instead of 24 hours, that’s a big hit to the business model. Maybe enough to make it impractical.

Your analogy doesn't really help because they aren't just "delayed". Money is wasted on incarcerating them, they come in close contact with criminals, and it makes their ability to get a job, lower.