My understanding (again, not a lawyer) it that this could have legal implications if you do something of your Japanese citizenship after reclaiming a foreign citizenship. Although apart from running and winning an election, or maybe holding a job restricted to Japanese citizens, I'm not quite sure exactly how you could get in trouble.
The loophole dual citizens (since birth) use is actually just that: since they never acquired a foreign citizenship (they are born with it), they never lost the Japanese one... so all Japan can do is order them to make unspecified efforts to renounce their foreign citizenship. Looking it up on the internet and concluding it's complicated seems to count as "efforts".
And if you lose your Japanese citizenship, you go back to being a foreigner. Without a visa. Meaning you risk losing your ability to live and work in Japan.
As most people who naturalize do so because they've established deep for-life roots in Japan, their Japanese citizenship is not something they're willing to gamble.