A durable record that maps votes to voters does not prevent voter fraud, it enables it.
1. Mobster goes in to vote but doesn't put it in the box - he takes the blank ballot paper outside with him.
2. He fills the vote on the ballot and gives it to the coerced voter 1. He expects a blank ballot back, or else. He has his goons watching the voter throw the real ballot in the box, or else.
3. Using the new blank ballot he goes to coerced voter 2, and the cycle continues.
Of course this is hard to scale.
I've been a scrutineer in a polling room. I could watch and challenge anything except the marking of a ballot. And I did. So did other scrutineers. The odds of the mobster keeping a rotating scheme without detection approach zero with great rapidity, especially since staying in the ballot after casting your vote is not permitted.
https://www-m.cnn.com/2016/10/29/politics/poll-watchers-moni...
Also, a voter who is being intimidated by a mobster could just walk up to a poll worker or police officer and report it.
A bit difficult since there are officials watching you put the ballot into the box. Or, gee, that's weird, failing to put the ballot into the box.
If there is no possible way for you or me to know (edit: prove) if I'm telling the truth, how is that worth your time and energy? It's not.
The benefit of a secret ballot is that it greatly reduces, if not removes, the incentive for coercion.