True, there's a lot of discretion in the Japanese prosecutorial system. Ditto USA. A big difference is that in Japan this discretion is mostly wielded judiciously.
Crime rate is also extremely low in Japan. I don't have evidence that the legal system is the reason, but I believe it's a significant contributing factor.
So where are all these righteously aggrieved innocents you worry about? They don't seem to show up in the stats.
[1] https://en.wikipedia.org/wiki/List_of_countries_by_incarcera... tldr: {jp:47, uk:163, us:693}
"Only 6.3% of the unnatural deaths in Aichi [prefecture] are investigated by a medical examiner, a minuscule rate even by nationwide standards in Japan, where an autopsy is performed in 11.2% of cases."
There are numerous cases of false allegations, and recently one guy who was being chased by the police despite claiming his innocence (and all his close friend and co-workers said that was not the kind of guy to do this) just fell to his death while trying to escape this situation.
My money is on the high degree of integration of the Japanese society. They are homogenous and everyone is well kept in place by their social network. That's why they don't become criminals as much, compared to US where crime correlates with disenfranchisement. The more a population feels the game is stacked against it, the more it will do crime, but an integrated society feels much more fair.