I keep a mental list of "things that are effectively impossible to backport onto an existing large source code base", which I really need to keep better track of. (By large, think developer-centuries at a minimum.) But it includes: parallelism/threads/concurrency, adding an external API, transactional correctness, ripping pieces apart and separating them by a network, and transitioning to a fundamentally different data store with different guarantees.
All of these have been done, of course, so they aren't literally "impossible". It's just that once you have a source code base with hundreds or thousands of developer-years into them, the amount of effort required and the resulting changes are so extensive that it's not entirely unreasonable to call the result a different code base. You certainly wouldn't end up with anything where you could code new functionality that works against both the old and new codebase without changes, unless you also made that a priority on its own and put a huge additional amount of work into it. (See, for instance, the amount of work PyPy has put into supporting CPython modules. Again, it's possible, barely, but it was on its own terms a very significant amount of work.) So there's a fairly real sense in which you didn't put these features into the code base; you created a new one with a clear parent line of development, but is no longer meaningfully the same.
Each of these things, and all the other ones I've forgotten, can be the basis of a new programming language, because at scale it's actually much easier to write a new language like, say, Erlang, and populate it with libraries, and proceed to use it to write tons of multithreaded, Erlang-robust project code than it is to write the same amount of multithreaded, Erlang-robust project code with C. However much work it was to create Erlang libraries, it's dwarfed by orders of magnitude by the amount of Erlang code using it. It's easier to write Rust and populate it with libraries and use it for projects than it is to write the same amount of safe project code with C. Etc.
Moreover, you missed my point, unless you're genuinely making an argument for radical ludditism.
I think what he was getting at is that the idea of languages integrating parallelism such that almost all code would be parallel all the time has proven to be a dead end. For the functional language research world promised for a long time that the main benefit for FP was automatic parallelism (no mutable states, you see). But it never really happened.
And in the more mainstream world, Java 8 invested heavily in parallel streams, but I've never seen an actual parallel stream in real code. I see threads all the time, despite everyone agreeing how awful they are. Parallel streams? Auto-parallelised FP code? No, doesn't happen. The level of parallelism is too small for programmers to think about. Most datasets are too small and most loops too unimportant to even take the risk of a bug creeping in through attempting to use parallelism, regardless of how convenient.
But that's obviously not what I was talking about. I wasn't even talking about parallelism in the context of language design. If an old tool is written in C and uses shared global mutable state so extensively as to make a performance boost from parallelism so impractical as to require a rewrite, then that's exactly the thing stopping people from "just" redirecting their efforts to improving old software. Because sometimes small incremental improvements aren't enough, and getting the buy-in to do a full rethink of an old tool takes tremendous social capital. And this is just about improving the old tool using its existing language---this isn't about going and using a new language with better support for parallelism. (Which, by the way, comes in many flavors.)
At no point did I talk about fancy parallelism related language features. At no point did I say that we should just go and add parallelism to an old program because that's what all the cool kids are doing. I framed it as a specific solution to a specific problem: making the old program faster. People like faster programs and speed ain't a fad. If you want to bring in examples of adding parallelism that don't improve performance, then great, but leave me out of it, because at no point did I advocate that.
I think your interpretation of tabtab's comment is a real reach, but whatever. Particularly given their other comments denouncing the evils of lambdas. Really? Give me a break.
True, sometimes it's good to have more than one way to do something, but sometimes it also creates a bigger and unnecessary learning curve. In the cases I looked at, making better OOP would result in a simpler language than adding lambdas, at least in my judgement. I'd be glad to debate it further, but this is not the best forum for such.
[1] Some argue the same languages should be used for systems software and applications development. I don't agree.
You've gone from "parallelism is a fad" to "let's subjectively evaluate language design based on my ideas of parsimony." Parallelism isn't some kind of language feature I'm touting. It's an implementation tactic that can be used to make certain programs faster. Before such tactics were common, it was easy to write programs that were blissfully unaware of it in such a way that it is difficult to bolt on later. I used parallelism as one possible example of something that would be incredibly difficult to add to older programs to combat the notion that we should stop re-inventing the world and instead improve older stuff.
Like most things in this world, parallelism can be misused. This is hardly interesting and really doesn't need to be pointed out. Moreover, the degree to which parallelism can be wielded effectively will, in part, depend on your tools. Some tools make parallelism easier to use or reason about than other tools do. With that said, even that was immaterial to my point, because I wasn't doing a comparative analysis of parallelism across programming languages. I was just using it as an example of a concrete improvement that one could make to older programs that is often not practical to do, precisely because they are older programs that are widely used. Parallelism isn't the only example of this kind of improvement; jerf provided other examples in this thread. But parallelism is an easy one to grasp because most programmers with a few years of experience can appreciate what it's like to refactor a large program that is deeply coupled to unsynchronized shared global mutable state to a program that isn't---which is generally a good idea if you want to add parallelism to it.
Your initial response to this was:
> I'm sorry, but the need for parallelism is exaggerated; it's mostly a fad.
But this is completely pointless. That parallelism is a useful way to make a program faster is taken as axiomatic in my comment. There is enough of my code (and others) out in the wild that uses parallelism to achieve some concrete measurable improvement that I feel it is obviously true and really doesn't need any further explanation. But here you are, nitpicking at an axiom with a bunch of nonsense about "fads," and completely missing my point in the process.
And I'm not against SOME parallelism support in app languages. It's just my experience that if you "need" to do it A LOT in applications, you are probably doing something wrong.
Re: "Like most things in this world, parallelism can be misused. This is hardly interesting and really doesn't need to be pointed out." -- Fads tend to greatly increase the % of misuse.
And lambdas and OOP do fight over similar territory, or at least can if the language has certain features. For example, Java coders may say, "I need lambdas here because OOP can't do what I need." But it turns out Java's OOP can't do what's needed, not something inherent in OOP.
Anyhow, English is often not sufficient by itself to clarify such debates; we'd need specific code samples and scenarios to explore.
Once again, you said:
> I'm sorry, but the need for parallelism is exaggerated; it's mostly a fad.
So, what's your point? Every time someone talks about parallelism---whether it's legitimate or not---you see it as your duty to pontificate on the "fad" that is parallelism? You've completely and utterly missed my point, which had nothing at all to do with debating the legitimate use of paralleism in a specific case. That there are legitimate uses of parallelism is enough to make my point.
> It's just my experience that if you "need" to do it A LOT in applications, you are probably doing something wrong.
Again, so what? This has literally nothing to do with anything I've said in this thread.
So, umm, thanks for derailing the conversation into your own pet cause?
> And lambdas and OOP do fight over similar territory, or at least can if the language has certain features. For example, Java coders may say, "I need lambdas here because OOP can't do what I need." But it turns out Java's OOP can't do what's needed, not something inherent in OOP.
Again, so what? This seems like a useless academic point. At no point did I invite such navel gazing.
Even if you had the person with the expertise, time and motivation to do the work to improve the implementation, you still aren't even guaranteed that the enterprise will succeed. You'll need immense social capital to even convince others that it's actually the right thing to do. Just think of all the new bugs that will be introduced. The angry users. The headache. The added maintenance burden and the pressure from trying to fix the new bugs quickly. It's an absolute nightmare and it's why it doesn't happen, IMO. It's also why starting anew is so much easier and so much more enticing. When you don't have any users, that's when you can really innovate.