All I have to do as a competitor to royally fuck you up is have a few people buy your product/s and then contact Amazon to complain that they are counterfeit or somehow not as represented by the listing. Boom! Listing shutdown, account suspended or both. If you call Seller Support for help the story you are likely to get is the the "Seller Performance team does not share the reasons for which they take certain measures". Boom! Again. You just got shafted, have no clue why and have no recourse. All you can do is wait and see what the internal Amazon process might decide.
I have a friend who was in a similar predicament. He was forced to have all of his inventory destroyed because it would have cost so much in shipping to recover it from Amazon warehouses that telling Amazon to destroy it was a better option.
It's a shame that Amazon can't just implement a weighting system with values based on how much a customer has purchased, how long they've had their account and the number and range of stars in reviews they've given. That would be a lot of map/reduce jobs over a lot of people, and reviews... but weighting the "overall" rating better would go a long way towards improving things for everyone.
I've seen studies that seen that (honest) ratings in number/star systems are strongly culturally correlated, and that American culture strongly favors giving the highest rating unless there are out-of-ordinary reasons to go negative, while some Asian cultures strongly favor giving middle to low ratings unless there is out-of-ordinary reasons to go positive.
So simple cultural correlations in awareness of and use of a particular brand of a product could result in very different review results with exactly the same customer satisfaction profile.
After reading your post I decided to get some more insight into this and you're definitely not the only one. It is somewhat sad, especially the amount of people that rely on amazon for shipping and deal with shipping reviews.
As for myself, I often skip SF/F or Programming books that are less then 4. But as you said, there are plenty of people who give 1-stars for almost no reason. This puts a ton more time into my background research of a product.