A genetic predisposition towards being afraid of predators. The assumption is that if you have 100 mice, and 50 of them show signs of being afraid of cats, then you breed those mice to produce another 100, noting the pairs that produced them. You can then see how many of the offspring are afraid of cats, and select that group to produce the next generation.

Some of the fear of cats will be learned behavior, some of it will be instinct. It could also be that the ability to learn the behavior has genetic links (think naturally smart mice). If you keep this cycle going, eventually you have a group that should be higher than your initial 50% fear rate. It's possible that you can't selectively breed for this, but it's also possible that you can. This is in some ways the inverse of how humans domesticated many different species.