You are forgetting the other side to evolutionary fitness: the environment. Human evolution has been progressively stagnated by the invention of adaptive technology. Being hard of hearing or nearly blind is no longer a deadly condition, hence it no longer prevents procreation, hence evolution doesn't factor anymore.
The human genetic structure is largely the same as it was hundreds if not thousands of years ago precisely because we can use tools to adapt to situations much faster than any evolutionary fitness can proof out and spread through the population.
As for the memory gene of mice, that would only have been important if a better memory made mice more successful at what they do. Evidently in the environments mice adapted to this was not the case.
"Of course evolution might have another "idea" of what is good for us than we do (evolution doesn't really care about us)."
Evolution doesn't have ideas. It does not use reason. It does not think. It exists purely as a pattern of change in species over time. There is no need to give it more credit than that, it is quite powerful enough without the addition of sentience.
Evolution is pretty random (or is it chaotic?). There are so many permutations that we have a DNA sequence that is very, very good for surviving in our world. It could be better though, and at this point in our civilization natural selection will not get us there.
Genetic engineering is inevitable. It will happen. In 10 or 20 or 100 years when it becomes practical there will be "conservatives" who are against it and "liberals" who are for it. They will fight for a while but in the end science will win, because science always wins in the long run.
Perhaps I am missing out big time by not using drugs. Some famous works of art are probably the result of drug use (how much of it, I don't know). I also remember a recent anecdote about a rock musician meeting Tony Blair and asking him how he manages his workload. Answer: "Certainly using different drugs than you" - of course it could just be "enthusiasm" being the drug, I don't know.
If you could become a world famous artist in exchange for living 10 years less, would you do it? Or whatever is your ambition (invent the Google killer, whatever).
I'd also like to take over Equatorial Guinea, but the current regime may fall before I get the chance.
A report from your friendly biomedical biologists.
Of course evolution might have another "idea" of what is good for us than we do (evolution doesn't really care about us).