Well, I suppose you could argue that the extra risk is the knowledge they have gathered as part of being a scammer. Knowledge being power and all that. A jaywalker doesn't know your phone number, where you live, and where your wife works.

Hopefully the scammer doesn't either, but hopefully this still conveys the point.

Edit: There's a difference too; jaywalking et al do not implicitly harm other people. I don't know all the legal intricacies, but theft (scams) is different.