I think there is a strong tendency of people programming in new languages to be overly convinced of the elegance and superiority of their new language as a result of (a) getting to rewrite software without the burden of those fussy, irritating existing users and (b) getting to 'rebuild' software without the rather exhausting process of feeling your way to an algorithm or system that never existed before.
There's absolutely nothing wrong with either of these things, but sometimes I think people working in hip new languages tend to mistake the elegance and superiority of their new language for the advantages conferred by a "rework". I bet I could build a pretty elegant regular expression system in C/C++ if I got to start from scratch, too!
This isn't a statement about one language being better than another, and I'm aware of how vexed a problem it is to compare new languages with old. If you built a pile of nonesuch systems in your new language people would complain that they can't compare these systems with stuff that they're familiar with. :-)
I'd just call for a bit of caution in assessing new languages being pressed into service for old, familiar tasks.
Sure. But note the OP used the word "irresponsible," so I'd say you're actually disagreeing with the OP, and not me. :-)
> I think there is a strong tendency of people programming in new languages to be overly convinced of the elegance and superiority of their new language as a result of (a) getting to rewrite software without the burden of those fussy, irritating existing users and (b) getting to 'rebuild' software without the rather exhausting process of feeling your way to an algorithm or system that never existed before.
Yes. But there must also be room for folks to seriously argue the benefits of one tool over another without simultaneously being dismissed as the zealotry of the newly converted, which is a feeling that I am all too familiar with!
Overall, I basically agree with you.
Indeed, one of my major complaints about the computer field is that whereas Newton could say, "If I have seen a little farther than others, it is because I have stood on the shoulders of giants," I am forced to say, "Today we stand on each other's feet." Perhaps the central problem we face in all of computer science is how we are to get to the situation where we build on top of the work of others rather than redoing so much of it in a trivially different way. Science is supposed to be cumulative, not almost endless duplication of the same kind of things.
What kills me about programming is the ugliness. In natural science, there's an "I know it when I see it" factor of essential simplicity and beauty. We may not know what to work towards, but we know when we've found it. It's hard to improve on F=ma, and it looks it.
In programming, we just keep endlessly tweaking. (Recently I commented here about a non-software job where I had no deadline, and the programmers here were legitimately concerned that lacking a deadline would cause me to "gold-plate" it.) Writing software, I'm having to use a language with 100 reserved words. There's no essential beauty here. It's a million little committee decisions, layered on the legacy systems of today. I guarantee that in 10 years it's going to be replaced by another language which has 100 reserved words, which will make slightly different design decisions, based on the legacy systems of that time.
Programming language design today is almost a random walk. I don't see it converging at all. Everybody takes a slightly different starting point, and then adds all the features from every other popular language. There's no science happening here.
If a fad is just a rediscovery (renewed interest in) some programming paradigm, they will all be familiar to you when they come back around. Hopefully your language incorporates it gracefully. I'm a little circumspect about how Java's lambdas turned out, but I'm not certain it could have actually been otherwise.
So, I don't think it is the inclusion of multiple programming paradigms in your language that is troubling, but rather how it incorporates them. One of the first pieces of advice (and very good advice) in Effective C++ is something to the effect of a) acknowledge that C++ is really a federation of smaller languages, and b) at the outset of a project, explicitly decide which parts the project will use, and which parts it won't.
From my personal view, very descriptive Lisp programs are actually quite easy to read - but they can be harder to maintain: there is this meta-level.
Lisp has a bunch of 'problems':
1) it has this meta-level where code is data and where programs transform code. This adds added complexity and increases the distance of the executing code from the written code. The machine is possibly transforming a statement before executing it, then the new code will be executed and can also be possibly transformed.
2) the code as data feature adds a layer of confusion: what is code and what is data exactly when?
3) the amount for programmer freedom makes it possible to write extremely hard to understand code. Especially the code might only be understandable while it is running (because then introspection and reflection can be used).
4) much of Lisp was developed at a time when more people knew how to use it. A lot of that practical knowledge is lost and thus it's difficult to educate new programmers. In the 'open source'/'free software' domain SBCL
OTOH, the fear of application/domain specific constructs is overblown. Sometimes groups report that Lisp code for large applications is much smaller and more readable than the equivalent, say, C++. If the code is full of low-level operators, repeating code, etc., then often the usual answers are configuration systems, extensive meta-architecture, added languages, an added scripting level, code generators, lots of manual labor, user-interface-level automation tools, ... this is no better or even worse than Lisp-level code generation/transformation.
Yes, but inventing new languages and building libraries for them has nothing to do with science. Nor is science even a motivator for these endeavours.
We're not talking computer science here. We're talking implementation details.
[1] Languages can and do implement one or the other poorly.
There's the weight of an existing community that also might make "improvements" more difficult. It's the same reason people leave large companies to go start smaller companies. Could they have stayed and made that company better? Maybe. Maybe not.
There's often a lot invested in the status quo.
Even if you had the person with the expertise, time and motivation to do the work to improve the implementation, you still aren't even guaranteed that the enterprise will succeed. You'll need immense social capital to even convince others that it's actually the right thing to do. Just think of all the new bugs that will be introduced. The angry users. The headache. The added maintenance burden and the pressure from trying to fix the new bugs quickly. It's an absolute nightmare and it's why it doesn't happen, IMO. It's also why starting anew is so much easier and so much more enticing. When you don't have any users, that's when you can really innovate.
I keep a mental list of "things that are effectively impossible to backport onto an existing large source code base", which I really need to keep better track of. (By large, think developer-centuries at a minimum.) But it includes: parallelism/threads/concurrency, adding an external API, transactional correctness, ripping pieces apart and separating them by a network, and transitioning to a fundamentally different data store with different guarantees.
All of these have been done, of course, so they aren't literally "impossible". It's just that once you have a source code base with hundreds or thousands of developer-years into them, the amount of effort required and the resulting changes are so extensive that it's not entirely unreasonable to call the result a different code base. You certainly wouldn't end up with anything where you could code new functionality that works against both the old and new codebase without changes, unless you also made that a priority on its own and put a huge additional amount of work into it. (See, for instance, the amount of work PyPy has put into supporting CPython modules. Again, it's possible, barely, but it was on its own terms a very significant amount of work.) So there's a fairly real sense in which you didn't put these features into the code base; you created a new one with a clear parent line of development, but is no longer meaningfully the same.
Each of these things, and all the other ones I've forgotten, can be the basis of a new programming language, because at scale it's actually much easier to write a new language like, say, Erlang, and populate it with libraries, and proceed to use it to write tons of multithreaded, Erlang-robust project code than it is to write the same amount of multithreaded, Erlang-robust project code with C. However much work it was to create Erlang libraries, it's dwarfed by orders of magnitude by the amount of Erlang code using it. It's easier to write Rust and populate it with libraries and use it for projects than it is to write the same amount of safe project code with C. Etc.
Moreover, you missed my point, unless you're genuinely making an argument for radical ludditism.
I think what he was getting at is that the idea of languages integrating parallelism such that almost all code would be parallel all the time has proven to be a dead end. For the functional language research world promised for a long time that the main benefit for FP was automatic parallelism (no mutable states, you see). But it never really happened.
And in the more mainstream world, Java 8 invested heavily in parallel streams, but I've never seen an actual parallel stream in real code. I see threads all the time, despite everyone agreeing how awful they are. Parallel streams? Auto-parallelised FP code? No, doesn't happen. The level of parallelism is too small for programmers to think about. Most datasets are too small and most loops too unimportant to even take the risk of a bug creeping in through attempting to use parallelism, regardless of how convenient.
But that's obviously not what I was talking about. I wasn't even talking about parallelism in the context of language design. If an old tool is written in C and uses shared global mutable state so extensively as to make a performance boost from parallelism so impractical as to require a rewrite, then that's exactly the thing stopping people from "just" redirecting their efforts to improving old software. Because sometimes small incremental improvements aren't enough, and getting the buy-in to do a full rethink of an old tool takes tremendous social capital. And this is just about improving the old tool using its existing language---this isn't about going and using a new language with better support for parallelism. (Which, by the way, comes in many flavors.)
At no point did I talk about fancy parallelism related language features. At no point did I say that we should just go and add parallelism to an old program because that's what all the cool kids are doing. I framed it as a specific solution to a specific problem: making the old program faster. People like faster programs and speed ain't a fad. If you want to bring in examples of adding parallelism that don't improve performance, then great, but leave me out of it, because at no point did I advocate that.
I think your interpretation of tabtab's comment is a real reach, but whatever. Particularly given their other comments denouncing the evils of lambdas. Really? Give me a break.
True, sometimes it's good to have more than one way to do something, but sometimes it also creates a bigger and unnecessary learning curve. In the cases I looked at, making better OOP would result in a simpler language than adding lambdas, at least in my judgement. I'd be glad to debate it further, but this is not the best forum for such.
[1] Some argue the same languages should be used for systems software and applications development. I don't agree.
You've gone from "parallelism is a fad" to "let's subjectively evaluate language design based on my ideas of parsimony." Parallelism isn't some kind of language feature I'm touting. It's an implementation tactic that can be used to make certain programs faster. Before such tactics were common, it was easy to write programs that were blissfully unaware of it in such a way that it is difficult to bolt on later. I used parallelism as one possible example of something that would be incredibly difficult to add to older programs to combat the notion that we should stop re-inventing the world and instead improve older stuff.
Like most things in this world, parallelism can be misused. This is hardly interesting and really doesn't need to be pointed out. Moreover, the degree to which parallelism can be wielded effectively will, in part, depend on your tools. Some tools make parallelism easier to use or reason about than other tools do. With that said, even that was immaterial to my point, because I wasn't doing a comparative analysis of parallelism across programming languages. I was just using it as an example of a concrete improvement that one could make to older programs that is often not practical to do, precisely because they are older programs that are widely used. Parallelism isn't the only example of this kind of improvement; jerf provided other examples in this thread. But parallelism is an easy one to grasp because most programmers with a few years of experience can appreciate what it's like to refactor a large program that is deeply coupled to unsynchronized shared global mutable state to a program that isn't---which is generally a good idea if you want to add parallelism to it.
Your initial response to this was:
> I'm sorry, but the need for parallelism is exaggerated; it's mostly a fad.
But this is completely pointless. That parallelism is a useful way to make a program faster is taken as axiomatic in my comment. There is enough of my code (and others) out in the wild that uses parallelism to achieve some concrete measurable improvement that I feel it is obviously true and really doesn't need any further explanation. But here you are, nitpicking at an axiom with a bunch of nonsense about "fads," and completely missing my point in the process.
And I'm not against SOME parallelism support in app languages. It's just my experience that if you "need" to do it A LOT in applications, you are probably doing something wrong.
Re: "Like most things in this world, parallelism can be misused. This is hardly interesting and really doesn't need to be pointed out." -- Fads tend to greatly increase the % of misuse.
And lambdas and OOP do fight over similar territory, or at least can if the language has certain features. For example, Java coders may say, "I need lambdas here because OOP can't do what I need." But it turns out Java's OOP can't do what's needed, not something inherent in OOP.
Anyhow, English is often not sufficient by itself to clarify such debates; we'd need specific code samples and scenarios to explore.
Once again, you said:
> I'm sorry, but the need for parallelism is exaggerated; it's mostly a fad.
So, what's your point? Every time someone talks about parallelism---whether it's legitimate or not---you see it as your duty to pontificate on the "fad" that is parallelism? You've completely and utterly missed my point, which had nothing at all to do with debating the legitimate use of paralleism in a specific case. That there are legitimate uses of parallelism is enough to make my point.
> It's just my experience that if you "need" to do it A LOT in applications, you are probably doing something wrong.
Again, so what? This has literally nothing to do with anything I've said in this thread.
So, umm, thanks for derailing the conversation into your own pet cause?
> And lambdas and OOP do fight over similar territory, or at least can if the language has certain features. For example, Java coders may say, "I need lambdas here because OOP can't do what I need." But it turns out Java's OOP can't do what's needed, not something inherent in OOP.
Again, so what? This seems like a useless academic point. At no point did I invite such navel gazing.
1. Even if re-implementation of libraries in new languages were free due to an excitement factor, real software is not built this way. New languages become old, and software needs maintenance. So what you get for free (supposing you're right) is not worth very much and not what counts, anyway. Most of the cost and value is in prolonged maintenance, which has to be done for multiple implementations in parallel.
2. It's reasonable to assume that work is never really free. But if we're hypothesizing about alternate realities, you may as well imagine that instead of new languages something else is invented to motivate you to do free work.
> real software is not built this way
This is a bullshit say-nothing phrase. You're using "real software" to presumably denote some dichotomy of software that doesn't exist. If you asked 10 people to define what "real software" actually meant, you'd get 10 different definitions. They might agree on the extremes; but there's a whole mess of crap in the middle that we could endlessly debate about.
> New languages become old, and software needs maintenance. So what you get for free (supposing you're right) is not worth very much and not what counts, anyway. Most of the cost and value is in prolonged maintenance, which has to be done for multiple implementations in parallel.
This isn't clearly true at all.
I mean, if you're going to talk about maintenance, then you also need to consider the impact of improved tools on the costs of maintenance. If a tool makes maintenance easier or less costly, then it's pretty hard to say anything definitive about long term costs. New tools can and do impact the usage of old tools, which in turn can have a direct impact on long term maintenance costs.
> It's reasonable to assume that work is never really free. But if we're hypothesizing about alternate realities, you may as well imagine that instead of new languages something else is invented to motivate you to do free work.
Absolutely! But that's not the argument I'm countering. I'm specifically countering the notion that new language development can be "irresponsible" because it "could" divert work away from other projects. What I'm saying is that it is not necessarily true at all.
There is a more measured argument to be made here, but the OP did not make it.
New ideas should be born and die in accordance with the resources of folks willing to keep them alive. We don't need some grand notion of "responsibility" to control the ebb and flow of ideas.
I think that what I meant was clear from context: software that has a long-term, significant impact.
> then you also need to consider the impact of improved tools on the costs of maintenance
Sure, but however much maintenance costs, it still needs to be done. It cannot be carried long by a novelty factor.
> New ideas should be born and die in accordance with the resources of folks willing to keep them alive. We don't need some grand notion of "responsibility" to control the ebb and flow of ideas.
I agree. Anyone should work on whatever they find interesting, and that (normally) doesn't make the choice irresponsible. But I still think that empirically new languages, on average, increasingly cost more and contribute less.
For me, Elixir was yet another language to learn, and was just a reimplementation of another language. But when I used it, I was like "whoa, this is sweet!"
You gotta build something new when you can't find something old that does what you wanna do.
But this is completely uninteresting. Nobody has claimed that transitioning to a new programming languages doesn't have a non-trivial cost.
You're countering a non-existent argument. I mentioned this in my initial reply to you: you're debating a point that is easy to refute. I don't know why, or where the source of confusion is, but you are.
> Moreover, productivity gains from new languages seem to have plateaued a couple of decades ago (except from some niches, such as system programming, that have seen little change in the past 20-30 years). I.e., we no longer see the same productivity boosts as we did going from Assembly -> FORTRAN, from FORTRAN -> C, or from C -> Java (and even then, the gains were smaller with each jump). Fred Brooks predicted in the '80s this would happen, and reality has shown even smaller gains than he had predicted.
This just sounds like a lot of opinion expressed as if it were a fact.
I was replying to the argument that the work is not at the expense of anything else because it wouldn't happen otherwise.
> This just sounds like a lot of opinion expressed as if it were a fact.
I think it's pretty well supported. The cost of developing software from scratch has not decreased significantly since ~2000 (there have been gains due to the availability of lots of open source libraries). In the mid eighties, Brooks claimed that no single language improvement would give us a 10x productivity gain over a single decade. While at the time his prediction was seen as overly pessimistic, it's been three decadades and we still haven't gained a 10x boost with all improvements combined (barring, maybe, the availability of open source libraries, which has nothing to do with language improvements). You can see clearly that companies are not migrating en-masse to some new language (as they did with assembly->C or C->Java/C#). Instead, many languages are being used, none so far seems to have a definitive advantage that encourages the majority of the industry to migrate. You do see plenty of claims, but no major bottom-line impact that causes decision-makers to say, "we must use that!" as was the case with previous generations, where, BTW, significant advantages became apparent quite quickly.
It's arguably overstated, but I'm curious if you would argue that there are languages created in the past 15 years that create as big of a productivity boost as Assembly -> C or C -> something with a GC?
Ahem.
> I'm specifically countering the notion that new language development can be "irresponsible" because it "could" divert work away from other projects. What I'm saying is that it is not necessarily true at all.
I (OP) agree that it is "not necessarily true", and my post deliberately made the very weak claim that "in some cases" "it could be irresponsible". I don't see how I could have made the argument "more measured" without eliding it completely.
> I don't see how I could have made the argument "more measured" without eliding it completely.
Exactly.
I actually KNOW who you are from your really consistently excellent work on useful libraries, whereas I have no idea who "pron" is and I suspect he is a troll
Now, is there an argument we'd be better off doing that exploration in a space designed to make DSLs easier than in building entirely new languages? That I dunno but it is worth considering.
Sending objects with references to other objects (which may need to be GC'ed) is even more difficult.
https://github.com/oracle/graal/tree/master/sulong
It's a part of Graal itself now. It runs C, C++, Rust, etc.
But first you have to obtain absolute top down control in order to do so.
Very similar to how command-and-control politics schemes could be in theory more efficient than messy democracies, but those messy democracies, while inefficient, produce variety that guard against the much larger errors of mis-allocation, ineptitude, or corruption.
Also the person with the vision of absolute top down control typically selects their vision as the correct one. If only we implemented my vision, since it is the correct one.
If we did what the OP suggests we'd be communicating with CORBA across the net, have artifacts such as "car" and "cdr" x10.
> but it's common for new languages to trigger reimplementation of, e.g., container data structures, HTTP clients, and random number generators. If the new language did not exist, that effort could have been spent on improving existing libraries or some other useful endeavour.
Yes, it could have, but would it? Speaking as someone who has done some of this reimplementation work in a newer language, I can unequivocally say that it would not in my case. I'm just one data point, but I don't think I'm unusual. There is an aspect of greenfield development that is appealing to me. It is a somewhat unique opportunity to execute a vision with a lot of freedom. There is also the benefit of new forms of expression that new tools give you. I'm painting with broad strokes here, but I'm generally a believer in the idea that tools themselves can both limit and empower the expression of ideas.
If it weren't for the new language, I surely would have done something else with my time. I don't know what it would be, but I know it would not be trying to expend the social capital required to make small incremental improvements to existing software using tools that I find too limited. As another commenter mentioned, I might have just watched more Netflix.