One possible answer to this question is the same as to the question: Why look at old paintings, read old novels and poetry, watch old plays, and study history? For me, it's partly because I am interested in what humans have created and thought in the past. It's partly because these works are often beautiful—an argument can be as beautiful as a work of art—and studying beautiful things is its own reward.
Whether they are true or not is another matter, but just as it's useful to read what other cultures in our own time think about a topic, it's useful to read about what people from other times thought. "The past is a foreign country; they do things differently there."
You can design a course of study that focuses purely on modern logic and epistemology etc if that's what you prefer. But philosophy is not a science (however much philosophers like to think it is) and its not maths. It is a branch of the humanities, and should be evaluated in similar terms to literary studies or history, not biology or physics.
In science (aka, natural philosophy) the goal is about finding answers. Perhaps in philosophy it is thought more important to ask questions, and the questions asked in "old primary texts" are still relevant today, and may be the exact same ones asked 2500 years ago.
What is [tT]ruth? What is beauty? Does God exist? What is the point of life? What is happiness? What does it mean to be a 'good' person? When it is okay, if ever, to lie?
Given your logic, which seems to be that we should only move forward, what's the point of studying history? Why bother reading Herodotus's The Histories when there are contemporary analysis? Tom Holland, who released a new translation, discusses some reasons:
So, there you have it, a (extremely reductive with huge gaps) history from early 1600s to the present, of philosophers building on each other's work into an intellectual framework you probably hold. There are of course many other branches; I just followed one, briefly, to the present to a tip that is probably pretty close to the thinking that informs your coursework.
Philosophy as a discipline makes more sense if you consider it as a form of literature. And it isn't true that "in no other field" beginners read primary texts -- literature scholars absolutely do this.
I'm a philosophy enthusiast, but I've never really understood the claim that philosophers seek to understand what's true. Mathematicians do that, and some types of analytic philosophers focus on that, but at least from my perspective philosophy is more about idiosyncratic excursions into interesting questions and ideas, the same way that fiction is about idisyncratic excursions into interesting plots.
IMHO no one rewrites Nietzsche because what would be the point? His headline ideas can be summarized in a rather short document, just like the plot of a novel can be summarized in a couple paragraphs, but it's the voyage that's the meaty, fun part, just like literature.
I think there are a few reasons:
1) I can't explain why, but, unlike physics or math, distillations and commentaries often miss key points and insights, and not uncommonly badly misrepresent works. Reading one then the other makes this clear. It's also the case that the canonical works are often (not always!) well-written, so difficult to improve on—possibly this is a side-effect of how philosophy works, as a field.
2) (I think this one's very important) Historical works of philosophy (which, see above, also have a tendency to be quite good for what they are) are, in themselves, a bit like exercises in math. Skipping wrestling with (at least a good number of) them is like trying to learn multivariate calculus without having yet put pencil to paper and worked through any exercises, for any kind of math, your entire life. In math, you have to do the work, to learn the material. Critically reading touchstone works in philosophy is philosophy's equivalent of doing the exercises, to a substantial degree.
3) The way philosophy has developed is kinda like rap: a lot of it won't make any damn sense—or, worse, will seem like it does, but actually you're getting entirely the wrong idea—unless you're familiar with the work of the person this track's responding to, and then that one is also part of a larger conversation, et c. Maybe that's a bad way for it to have developed, but that's what we've got.
1. I think you misunderstand me. I am not interested in distillations and commentaries. I am interested in building upon the works of past philosophers, improving their weak arguments, supplying better arguments that they missed out on, finding better examples than what they used, and finally going past what the said, and saying even more. The replacement of Nietzsche's books should be books that say somewhat similar things, but which are better arguments than Nietzsche. This must be possible for any intellectual discipline, because any such discipline by definition must create transferable knowledge and skills. And if you are not, the problem cannot be with the character of your intellectual discipline, but with the social structure and practices of your discipline.
2. To follow upon point 1, and to answer your points 2, take the example of GR again. What Einstein wrote is simpler and probably easier to understand than good modern graduate textbooks on the subject - the latter have far more sophisticated arguments, and they go explore the area far more than Einstein ever did. Because of course, physicists and mathematicians kept building and improving.
Similarly, what I expect is that philosophers should constantly be making better and more sophisticated arguments, that require more and more skill and acumen to understand. But a beginner starts with very carefully constructed arguments that easy to understand, yet astounding, and as her skills increases, she is given more and more difficult arguments to grapple with. this is how things happen in other intellectual disciplines by design. But not in philosophy, and I truly don't understand why.
The "replacement" for Nietzsche's books, if that's a goal, must also address WHY Neitzsche's arguments are weak or insufficient. It must also address, to a degree, structuralism/post-structuralism, postmodernism, etc. This is not a process wherein Nietzsche (or Heidegger, or Hegel, or whomever). Questions of "how ought one live?" are not narrowly focused. Simply because you did not study philosophy and seemingly disregard the idea that different people may adhere to different ethical systems (as one part of philosophy) does not invalidate the discipline. There is no universal, provable "right" answer to "what is ethical?" Certainly there is overlap between virtue ethics and utilitarianism and intuitionism for how one might behave, but the reasoning behind them all differs.
Humans are not omniscient. Philosophers are not omniscient. It's disingenuous to talk about general relativity as an example, and much more cogent to talk about theoretical physics (to which I can't speak with any authority except as a lay reader who's aware of multiple theoretical frameworks which cannot be simultaneously true). Sure, you can have "more sophisticated arguments (mathematics)", but the answers to some of these questions may never be known. In philosophy, we will NEVER know the answers to questions about the human condition, the nature of knowledge, etc. Just because it's not mathematics which are "transferable" and can build on top of each other does mean that the knowledge is not transferable or that the arguments do not get more sophisticated.
Exactly what you're describing DOES happen in philosophy, and has always happened.
Take an idea like Newton's Law of Gravity, and let's separate the words that Newton wrote, and the content of the ideas behind those words. Many people, over decades, have considered how to best to present and teach the content of the idea in the best way possible. The same happens with Einstein's General Relativity.
Current practicing astrophysicists, even those whose career is making measurements of General Relativity, have very likely never read neither the original words of Einstein nor Newton's words on Gravitation. This is not a scandal because the physicist's attitude is: who cares about the words used in the original description, when what matters is the content of the idea?
Further, if smart and pedagogically minded people can work and re-work the actual content into a much more understandable distillation of the actual idea, why not use that? Everyone still realizes that it's Newton's idea.
Contrast this with a philosopher working on some modern topic that built on some ideas of Nietzsche -- saying they have never read the original words would be seen as an admission of professional neglect. But even a 1st year undergraduate, what's taken as an important thing is to read the actual words of Nietzsche (perhaps in translation) and not someone's presentation of the ideas in their best possible form.
So, let's circle back to the original distinction: the idea and the words originally used to express that idea. In physics, you attempt go to the best exposition of the idea (which is rarely the original words). In philosophy, there is a primacy given to the words themselves that is bizarre from the physicist's perspective.
Finally to address what you wrote here:
> The "replacement" for Nietzsche's books, if that's a goal, must also address WHY Neitzsche's arguments are weak or insufficient.
This isn't what is being asked for. Easier perhaps to shift into the physics example.
When learning of Newtonian Gravity you learn about the concept in the best form possible (again, not the original words used by Newton but a modern presentation of the content of the idea). That's the equivalent of what's being asked for w/ Nietzsche.
You may later learn how Newton's ideas are incorrect just as you may later learn ways in which Nietzsche's arguments could be improved, but that's for later.
The missing idea in philosophy is having the best presentation of the content of the idea.
Long answer: First, although there has been progress in philosophy, it is much slower than in the natural sciences and harder to discern. Second, unlike any other discipline, philosophy has no agreed upon methodology and no agreed upon subject matter.
A consequence of both of these properties of philosophy is that older works can stay relevant and transformations and interpretations of them are often contested. It is very common for secondary literature to re-interpret heavily and in ways that might not capture the original author's intentions. Since work in philosophy is primarily working with texts, this explains the heavy focus on primary literature.
Since there is plenty of progress in philosophy, however, just reading the primary sources mentioned above would not get you anywhere. As a rule of thumb, the older a book or article, the more you'll have to take into account their reception. Otherwise you won't philosophize at a contemporary level and won't be able to publish about the work. For instance, writing anything insightful about Kant's Critique of the Pure Reason or Plato's Politeia will require specialization and taking into account many previous interpretations.
Some people would dismiss philosophy entirely because it has no common subject matter and widely accepted methodology, except for having relatively high standards for working with texts that are similar to those of historians and fairly high standards for laying out arguments. That's fair enough, but kind of short-sighted. Philosophy has had a tremendous influence on the sciences and still occasionally has.
Basically, philosophy is all about reading texts and unfortunately this often gets in the way of actually "philosophizing." I've met many philosophers who got lost in texts and never returned - they end up doing history of philosophy.
My best advise to anyone interested in philosophy is to approach it the way I was taught in Tübingen 30 years ago. Approach original texts seriously and in-depth, but carefully choose those texts and only read what interests you. There is no need to read the Critique of the Pure Reason if you're interested in contemporary analytic "accuracy first" epistemology, for example.
This is my general criticism of philosophers [2] and the way they have constructed their discipline. In physics/biology/economics/etc a super smart person comes along and does some excellent work. Then later on, other smart people learn the stuff and build upon it, and rewrite it to make it better. Modern textbooks on General Relativity are an order of magnitude better than what Einstein wrote. Why has no one understood Nietzsche, and written a better and more understandable version of Nietzsche's work? And why is everyone not reading that?
[1] I have read some/part of the books you mention.
[2] As a physics phd, I have engaged in and enjoyed and found useful practical philosophy around issues of epistemology.