Choosing to make your book "fit" is often a good call. You're actually going to attract people who know roughly what they want. My mother reads a lot of trash romance, it's not very important who wrote it, but a handsome guy on the cover signals that this is roughly the right book, the attire and surroundings are a vague gesture at the sub-genre, guy in a white coat? Medical. Business suit? Maybe 1980s.
There's no rule against refusing to obey convention, "Twenty Jazz Funk Greats" is, in fact, not Jazz-funk, and indeed didn't have twenty tracks. But nobody working on that album thought "This is a commercial route to success". It's art. Really difficult to listen to art. If you are not making art but have some other purpose you probably do want to signal your intent to people merely browsing.
For example, here are some images from Library of Congress with crabs in them that I think could make for interesting book covers:
- https://www.loc.gov/pictures/resource/fsa.8a23704/ "Residents of Raceland, Louisiana, eating crabs at crab boil" (1938 Sept.)
- https://www.loc.gov/pictures/resource/jpd.00748/ "Benkeigani to tsubaki" (between 1825 and 1830)
- https://www.loc.gov/pictures/resource/ppmsca.69838/ "Frozen crabs at a restaurant" (January 1954)
- https://www.loc.gov/pictures/resource/highsm.14204/ "The Barking Crab restaurant on the Fan Pier across the Northern Bridge from downtown Boston, Massachusetts" (between 1980 and 2006)
- https://www.loc.gov/pictures/resource/ds.00242/ "Cancer" (latin name for crab, you'll probably want to edit the image to avoid any unfortunate connotations) (1825)
But before picking any of these or other images from the Library of Congress, check the "about this item" link and see in particular the "Rights Advisory", including any links to further details about rights and usage in that field, and keep also in mind things like what is pointed out in one of those links; "Privacy and publicity rights may also apply". IANAL, TINLA.
a tasteful public domain image of a crab or something like hundreds of years old but then detailed enough kind of like one of those O'Reilly's books and then make the whole cover white with some text that stands out... It has all been done before and I anal but I doubt O'Reilly's can copyright or trademark that design feeling.
Edit: unless the authors are trying to crowd source the proof reading and peer review by releasing an early access copy? This cover draws attention from the average reader who is drawn to comment about the book because there is something to criticize? Maybe this is some kind of advanced 3D chess?