Not recognizing this (like in the article) is a clear sign of a single-paradigm developer that never really understood anything else but is too arrogant (insecure maybe?) to assume he doesn't understand something. So, the problem must be with everybody else.
Anyway, a funny red-flag is that reaction to Haskell IO. People that never really learned Haskell tend to complain about IO with a deeper complaint of "why to I have to keep using monads?", while after people learn it they tend to complain about the lack of monad composition, with a deeper complaint of "why can't I use monads for everything?"
But for solving several of them, you need to compose your monads, and there is no good general way to do that. There are context dependent ways to compose some of them, but you are always thinking about their compatibility and they often require that you specialize your code.
It's (provably, I think) impossible for fully general monads, but for traversable-but-otherwise-general monads you can do[0]:
newtype (%) (f1::k1->Type) (f2::k2->k1) (a::k2) = Comp { unComp :: (f1 (f2 a)) }
instance (Monad f1,Monad f2,Traversable f2) => Monad (f1 % f2) where
join (Comp a) = Comp $ a >>= (map join . mapA (unComp ))
(Comp a) >>= k = Comp $ a >>= (map join . mapA (unComp . k))
(For that matter, you only need f2 to be traversable; f1 can be any monad at all.)0: copied from my own code, so it might need other adjustment than s/map/fmap/ and s/mapA/traverse/.
If dealing with Untrusted File Formats perhaps you should use a tool purpose-built for Wrangling them Safely, WUFFS.
You won't find a Hello, World example for WUFFS because Hello World prints out text to your console, which is exactly the sort of nefarious stuff bad guys might try to do and so WUFFS doesn't even provide any mechanism you could use to do that even if you wanted to. But it does Wrangle Untrusted File Formats Safely, shrinking your remaining problem space.
For example WUFFS would be appropriate for taking files your users claim are JPEG "photographs" they uploaded for their "profile picture" and turning each one into either a 64x64 pixel RGB array or an error without any risk that they seize control of your profile picture program and do goodness knows what else instead.
Although Rust's memory safety means you can achieve confidence a "photograph" doesn't corrupt memory it doesn't require the rigour WUFFS brings to file parsing, so a Rust program could end up confused about unforeseen state while parsing the file. For example in Rust you can write a function that might mistakenly overflow a 32-bit integer and, in production it will just silently wrap. In WUFFS that function won't compile until you either decide explicitly what should happen (e.g. wrapping, saturation) for each overflow, or you trap all the cases where it could overflow as an error. This is very annoying of course, but we're parsing Untrusted File Formats and if we leave anything to chance that will be exploited.
I'm pretty confident that I can parse untrusted binary data in Rust with nothing worse than a denial of service. (And I have over a billion 'cargo fuzz' iterations to prove it.)
But WUFFS is even more opinionated and strict than Rust, and so it can offer even stronger guarantees. Admittedly, it looks pretty obscure and the documentation is a little light, but it's a great idea.
I am really just done with CVEs, or at least the sort of CVEs that appear in C programs. We know how to prevent so many classes of security holes completely, allowing us to focus on the harder challenges.
You can do this in Rust, up to a point. There's a lint to ban dangerous arithmetic: https://rust-lang.github.io/rust-clippy/master/index.html#in... . You can then use the {saturating,wrapping,checked,overflowing}_{add,div,mul,abs,...}() methods to decide exactly what should happen on overflow.
But WUFFS seems a lot nicer. Judging by the README it first tries to determine whether overflow is actually possible, while Clippy will happily forbid you from running "1 + 1".
Nice in theory. In practice a manager will ask you to build A, then later they will ask you to add B and C, and these components should interact seamlessly of course. If you chose your language based on A, then you might get stuck on B and C.
- "Our web apps get written in Typescript."
- "Our data science gets written in Python."
- "Our inner loops get written as Rust CLI tools that contain no business logic." (Or whatever. It depends on the problem domain.)
This works because TypeScript, Python and Rust are all very good at certain tasks, but they're each also fine general-purpose languages with lots of third party libraries. If you do somehow wind up writing a webserver in Python, you'll be fine. Of course, something like Haskell is a deeper commitment with more tradeoffs, and it's the wrong answer for most organizations. (But if I were mathematically modeling the pricing of complex derivative contracts, Haskell would make the list.)
But this is why good senior people with taste are so useful, and why competent technical management is worth its weight in gold. Sometimes you want a toolbox with 2 or 3 well-chosen, versatile tools. And sometimes you get organizations that try to pound in nails with a screwdriver.
Our web apps get written in Perl (later added: with jQuery for FE interactivity)
Our data science gets written in Oracle stored procedures
Our core business logic is written in Oracle stored procedures!
Fat clients are written in Smalltalk
These were arguably even sort of well chosen at the time. (I won't say which company this was as that would identify me and I changed some details but this is a real life example)
For example I've seen:
- COM and ASP.Net to Java and JSF (general web apps)
- Forté 4GL (look it up, it was very interesting) to J2EE (general enterprise software)
- MIDP Java to React Native, following mobile phone evolution (mobile app front ends of general enterprise software)
- HTML frame messes to Java applets to JavaScript interactivity (high profile public web site)
- ColdFusion, or something of the sort, to Sharepoint (another high profile public web site)
- SharePoint to a BPEL workflow engine (important workflow-oriented application)Not using COM for native programming on Windows is being stuck with Windows XP.
Was this mad? Yes. Was it quicker to market than a rewrite? Yes.
I've done a few total rewrites now, and the only one I am sure was a good idea was for this reason (DPX, the Java re-implementation of Garlik's core software before it was purchased by Experian Consumer Services many years ago).
Pretty much every app, driver, and framework I've worked on across the web, desktop, and mobile has been mostly glue holding different components together. Choosing a different language would require a nearly-complete rewrite in every case except for a tiny section of independent interesting logic (and of course, if you arbitrarily picked some meme language for your "business logic" you'd have to write more glue code to attach it to the rest of your software..)
Anywhere you're e.g. dealing with an eventually consistent database... that is not business logic, that is all glue code that exploded due to scaling concerns.
But people have to get mad about the one throwaway comment at the top without even reading the next paragraph, let alone the rest of the article :(
Of course, using a formal methodology like B, ceaseless, flawless vigilance is mandatory.
[0] https://www.cambridge.org/gb/academic/subjects/computer-scie...
How so? You can do whatever paradgim you want in CL. There is no policing involved. Not in the sense of Haskell or Rust at least. CL will let you modify globals or share data willy nilly just fine.
Mixing functions and values in lists is not really a restriction either, it opens up for all kinds of shenanigans.
- You should be able to implement Lisp-in-Lisp within a few pages of code. This a profound and remarkable design constraint. (See http://languagelog.ldc.upenn.edu/myl/llog/jmc.pdf (PDF) for an introduction.)
- Syntax would only hide the fact that code is a simple data structure that can be manipulated like any other data structure.
Just like Rust's strong opinions buy you stress-free threading and bit-banging, Lisp's opinions ensure that it's an almost ideal environment for inventing domain-specific language extensions.
But Rust is poorly suited to object graphs where everything mutates everything else, and Common Lisp is poorly suited to proving that a large program obeys certain strict rules.
I love opinionated tools, but I try to choose ones with the "right" opinions for the task at hand.
As I'm sure you know, a Lisper would probably design a domain specific language that was well suited for such proofs. One of the projects I'd like to get around to one day is a Lisp hosted general purpose DSL[1] based on predicate transformer semantics[2] that combines writing specification and program text into a single interactive experience. Think something along the lines of smartparens[3] strict mode, where the editor doesn't allow you to make an edit that will violate the count("(") == count(")") invariant and you modify the tree structure with operations like slurp and barf, but a bit more general.
[1] Haha, I know.
[2] https://en.wikipedia.org/wiki/Predicate_transformer_semantic...
Even if you do need to model an arbitrary graph (for example), though a traditional programming language would make it a lot easier than in Rust, I still wouldn't necessarily let that dictate the choice of programming language. Most likely your program will end up doing a lot more than just banging on the graph and those parts may benefit from a more disciplined programming language.
This is particularly relevant for certain algorithm, such as atomic initialization, that get significantly more complex with in-line ownership tracking.
http://db48x.net/reposurgeon/rust-port-docs/reposurgeon/path...
Also, because of Rust’s wonderful type system it will be fairly straight forward to replace the string keys in this code with interned strings instead. (The amount of strings we use for paths in a large repository is pretty astounding.) That might now be almost possible in Go, if I learn the new generics.
Certainly not everything is a good fit for Rust’s ownership model, but honestly most code benefits from it at least a bit, and steadily more and more is being figured out about how to mesh other models into it without too much pain. It’s very liberating, protecting from various mistakes that are too easy in other languages (and that’s specifically where I miss it).
Ownership modelling can require more effort initially if the problem doesn’t match Rust’s capabilities very well (or put another way, if the problem doesn’t match how hardware works without garbage collection), but subsequently it’s liberating to not need to worry about various problems that are ubiquitous in other languages.
I reckon it similar to the debate of static versus dynamic typing, though more subtle. For toys, dynamic typing is adequate and static typing a burden, but as you scale a system up, dynamic typing makes life harder and requires more discipline to avoid ossification or drowning from technical debt. It’s taken time, but there has been a significant resurgence in static typing as people slowly come to realise its value through bitter experience in dynamic typing. Like static typing, a strict ownership model requires more effort initially, but makes life easier down the path, and I expect that at least the basic concepts of it will be picked up in more languages new and old over time (this has started already, actually, with Swift and D springing to mind), though it’s probably a harder concept to beneficially retrofit to a language than static typing.
You may want to mutate and forget about ownership, but Rust doesn’t let you do this, and it’s for your own good ;-). In time you get more of a feeling of why it is the way it is and learn to appreciate it.
And if the abstraction bedrock is high, then problem domains below that bedrock can't be approached with the language.
Various Common Lisp implementations like ABCL (Common Lisp on the JVM), ECL, CLASP, mocl, ... don't support saving and starting images.
> And tree-shaking is less effective because the language is so dynamic it's hard to guarantee some code won't ever be called.
That depends on the delivery system. For example in LispWorks I can manually remove a lot of functionality:
http://www.lispworks.com/documentation/lw71/DV/html/delivery...
[0]: https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
Another one was CLICC : https://github.com/hoelzl/Clicc which has a language definition for CL0 : https://github.com/hoelzl/Clicc/tree/master/doc
There are or were a bunch of such small delivery oriented implementations: Oracle bought one years ago, Gensym has one, mocl is derived from CLICC, ...
L was actually a subset of Common Lisp. It would run pretty nicely on a machine with 1 MB total RAM and a 16 MHz CPU. All the basics were there: lambdas and macros and symbols and modules, and they all worked how they did in Common Lisp. (Or they were close enough that you could write and test in Common Lisp and then cross-compile.) There was a REPL that could run most code, and an ahead-of-time compiler.
It was a fantastically nice environment for embedded systems too small to run something like Alpine Linux. You could compile a program, load it onto actual hardware, and then tweak things interactively using the REPL. And you could invent all kinds of macro DSLs.
Of course, all this becomes moot once your system is large enough to support Alpine Linux and an SSH server. If you can run Linux, then you can use Lisp or Python or PLT Scheme or Rust or anything else you want, and you have half a million open source libraries available.
Still, L is proof that Lisp is a great embedded language for anything with a MB of RAM. You can have a pleasant development environment and tons of flexibility, with an excellent REPL.
The idea of CLICC was to compile programs to small static C programs. Really small and with little or no dynamic features of Lisp (code loading, runtime compilation, eval, redefinition, ...). It did not have all the bells and whistles of ECL (virtual machine, runtime code loading, debugging, full dynamic Common Lisp, ...).
But say you do all that work. Congratulations, you now have an ecosystem of one person. To do anything in this language you need to reimplement a standard library plus third-party libraries, or at least write Rust-type-safe glue code to existing Common Lisp libraries. If someone wants to join your project, they'll need to learn your DSL first.
When you write a bespoke DSL for a problem already solved by a programming language with a sizable community you're giving up a lot of network effects. That is something the OP completely failed to account for.
It's a lot easier than writing Rust.
> you now have an ecosystem of one person
Do you think it would have even been possible for one person to write Rust? Imagine if all the effort that went into writing Rust -- and Haskell and Swift and C-sharp and Python and Java and Javascript all the other myriad balkanized languages that have come along since CL was standardized -- had gone instead into improving CL. Do you really think we would be behind where we are today?
Inventing a new and different syntax for every new language you design is a choice, not a technical necessity. The fact that languages have communities is survivorship bias. It's a bug, not a feature. In Common Lisp, it is possible for a single person to write something truly useful. In other languages, it is not. This is not to say that Lisp's lone-wolf culture is desirable; it's not. But that is NOT a reflection of any technical problem with Lisp. It's a political and cultural problem. And the first step to solving that problem is getting more people to recognize it for what it is.
No, but I don't think it would be possible for someone to implement an S-expression Rust dialect on top of CL either.
> Do you really think we would be behind where we are today?
Hard to say. The counterfactual world where everyone decided to build on top of CL is so weird it's hard to say anything about it. Also, "improving CL" is an unclear term. For example if you want to replicate all the benefits of Rust by "improving CL" then that means, among other things, you need an implementation of "CL" that can compile your Rust-equivalent subset/superset of CL to bare metal with no runtime (including no GC). That implementation will take advantage of Rust ownership and other properties thus will not compile regular Lisp code. If you have a separate implementation for a special dialect of CL with distinct static and dynamic semantics from normal Lisp, it is only very loosely still "CL".
> In Common Lisp, it is possible for a single person to write something truly useful. In other languages, it is not.
This is an absurd overgeneralization that makes you sound like a nut.
You're right. Allow me to rephrase: in CL it is easy to write a compiler for a new language because you don't have to write a parser or a back-end. It is so easy that it is a plausible task for one person to write such a compiler and make it truly useful, and indeed there are several extant examples. In other languages it is much harder.
This surely cuts both ways: if you write a DSL that adds the constraints and provides the guarantees that you need, you've re-implemented big parts of Rust or Haskell.
Or maybe you only need a tiny fraction of the constraints and guarantees that they provide. But it's likely that you'll need more over time, and then you wind up with not just a re-implementation but a gradually accreted, rather than coherently implemented, one, and that's unlikely actually to provide the guarantees it's meant to provide.
Maybe. Or maybe all I had to do to turn CL into Haskell is implement the Hindley-Milner algorithm. I didn't have to write a parser or a compiler because I can just re-use those from CL.
There's a number of issues with that:
- I'd be missing all the optimizations that can be performed due to purity.
- There's more to Haskell's type system than just vanilla Hindley-Milner, and the implementation of it isn't particularly trivial. https://github.com/stylewarning/coalton is the closest thing and it's still missing a large amount of the type system.
- Doing the implementation would be a significant amount of work to get it to integrate well with the language, and it would be a layer tightly glued on top instead of integrated with the language. I've seen many good DSLs embedded in lisp, but a type system is hard to embed in any language because it changes fundamental semantics of the language. Typed Racket is a massive project and it's lacking things like ADTs.
- A major part of Haskell is the standard library, a good chunk of the semantics of Haskell people use on a day to day basis, like monads and etc, are a part of the standard library.
What I'm advocating here is not retrofitting Lisp but embedding DSLs within Lisp. In a DSL you have complete control over the semantics, and so you can do all the optimizations that Haskell does.
> coalton is the closest thing and it's still missing a large amount of the type system.
Note that this was written by one person. You have to distinguish between what has actually been done given the current state of the world and what would be possible if people made different choices. If a tenth of the effort that has gone into implementing Haskell had instead gone into implementing a Haskell-equivalent embedded in CL, that effort would plausibly be competitive with actual Haskell if not superior.
Not a fun codebase.
Now they are reimplementing some of those things for performance reasons, but you're going to have a hard time convincing anyone that a lisp-based rust you wrote is going to be more performance than the one that exists today, so they'd need to make those optimizations anyway.
You are absolutely right that I'm squeezing a lot of content into the word "just". But I'm not doing it in a vacuum or out of ignorance.
It's such a wildly ridiculous claim though. It implies either
1. that the syntax of Haskell is the majority of the work, such that using a dsl would remove 90% of the effort.
2. Somehow a dsl for Haskell embedded in a lisp would be more efficient to work in, so much so that development would be an order of magnitude faster.
1 is absolute nonsense. 2 might be true in a sense. I doubt it, but even if it were, such a language would no longer be Haskell, so you'd lose out on the value you gain from it's syntax. The same is true for rust.
S-expressions are nice, but they aren't always the best way of structuring things, and if you're going to forego them to better embed another language, why constrain yourself to the lisp runtime?
The more I look at this, the less sense it makes.
You can create a DSL for a team working on a specific problem. Lisp is good at that. What I have not seen is a DSL that anyone else wants to use.
Someone builds a DSL, a small team uses it. It just has to meet the needs of that team. It doesn't have to be bulletproof or elegant. It can have all kinds of rough corners. It can take undocumented tribal knowledge to know how to use it.
The comparison to Rust is therefore almost completely misleading. It's like comparing Linux in 1992 to a commercial Linux distribution today.
Yes, you can't remove the constraints of Rust or Haskell (other than using unsafe, I suppose). But if those languages have the constraints that you want, then trying to write that as a DSL instead is... probably hubris. Also wasteful.
It would be a lot easier than inventing Rust, for two reasons:
1. You don't need to write a parser.
2. The target language for your DSL compiler can be Common Lisp rather than machine language without sacrificing performance.
Those two things make it absolutely certain that however much work it is to re-invent Rust in CL it will be strictly less work than inventing Rust from scratch.
There are other benefits to this approach as well. For example: if, after implementing your DSL you discover that you've made a sub-optimal design decision somewhere along the line, it will be a lot easier to tweak your DSL than it is to tweak Rust.
One of the things people really like about Rust is its fantastic error messages. They're very helpful.
But, having decided you "don't need to write a parser" you're in a tough place for your DSL since there are problems where Lisp's parser will just tell your DSL programmers something went wrong here and leave them clueless as to what exactly is wrong in their program. Rust wouldn't have done that, but of course its compiler owns the parser too so it gets to handle all the corner cases correctly.
You could even write a FPS using OpenGL where shooting functions, or pieces of suctions removes them from the current compile job (or even deletes them from the source code entirely). In a macro.
† Or of course, to deliberately not emit descriptive error messages, e.g. whichever-compiles!
And yet rust has been invented, and it seems the CL version is hypothetical.
And in a world where Rust already exists, in most cases it's easier to just use Rust than trying to write "that kind of thing" as a DSL in Lisp.
Uh, how exactly does that work? Which CL impls can e.g. vectorize as well as rustc does?
Note that static type checking can be difficult to work with when using the prominent interactive programming style since the types aren't always wellhdefined and/or in flux while writing and adding new bits and pieces of code, or redefining types, adding struct/object members at runtime, etc.
But it's possible, yes.
> In my experience, Commony Lisp is a deeply opinionated language, and its most dramatic opinion is that "your code and your data should have identical representations and structures." And for the right problem, that restriction is extremely powerful.
Code in common lisp is almost entirely in a tree structure via nested linked lists. However, when programming in Common Lisp, I don't use this specific data structure very often for things that aren't code. Furthermore many other Common Lisp programmers write software this way as well, which is a bit of a counterexample to the assertion I was replying to.
[edit]
I think an opinion common lisp does have is that code should be easily manipulable as data. As much as scheme programmers like to call CL macros "unhygienic" it's far easier to write a correct general-purpose CL macro than a correct macro in other languages.
Yes, but oftentimes this is more like:
"If we tie you to this bed, you will never have a car accident! You'll also never get lost! This also makes it easier to stay healthy as we'll be feeding you the best food!"
Sure, but I'll also miss all kind of activities I could be doing outside on my own. I'll also get bed sores. And I often want a steak or ice cream, even if it's not ideal, but you insist bringing me this organic gourmet vegan crap.
No one sets out to create programming languages that tie people to their beds. If people add restrictions, it's not to make users miserable.
It's fine if you like being unrestricted, I like that too.
However, when you see people going out of their ways to add barriers to their own work, you should assume they reap a benefit that you're not fully appreciating, not that they hate fun and want to outlaw ice cream.
People can still end up designing such languages, even if they didn't set out with that specific goal.
(Kind of like how you can bring war and division, even if your goal is to bring some religious peace on Earth or "the revolution").
>If people add restrictions, it's not to make users miserable.
That's already explicit in my comment though. They didn't tie the person to his bed to make him miserable but to spare them from car accidents, to help them never get lost, and other such niceties!
The misery is a by-product, not a design goal.
They mean well. Fine. I still don't have to like it, and I don't have to accept it.
For example, as have been shown for example countless times, C programmers are unable to properly free allocated memory so solutions were made (GC, or rust’s ownerships). It is egoistical to think
The point is, you don't have enough information to choose for me. You can't do it. Even if you can absolutely, 100% correctly decide what the "right thing" is in any given situation, you don't know all the situations.
(And if you'd say "the right thing is to not use Pascal in that situation", I'd agree with you. But as a language designer, your language will get used in places where even you may think it's not a great fit.)
Yeah, it might suck that you can get a ticket for treating a red-light like a stop-sign at 6am in the middle of no-where. Though this is me speaking from a "likes rust" perspective.
Rust has unsafe, Haskell has unsafePerformIO. You might not like how verbose some of the stuff is but all these languages give you a foot gun if you really want it.
A well known article, written in bad faith, is "Why Pascal is Not My Favorite Programming Language".
http://www.cs.virginia.edu/~evans/cs655/readings/bwk-on-pasc...
Why bad faith? Most of the critic he makes against Pascal was sorted out by Pascal dialects.
Which one can consider doesn't count as dialects aren't the main language, yet by 1981, C was mostly available in K&R dialects outside UNIX, like Small C.
And besides those Pascal dialects, Modula-2 standard was released in 1978, while Mesa was powering XDE and Bravo at Xerox.
That reminds me of Yes, We Have Noticed The Skulls [0].
[0] https://slatestarcodex.com/2017/04/07/yes-we-have-noticed-th...
Arthur Laffer is an economist. Laffer has helped provide advice to numerous Republicans and Republican projects which of course didn't achieve their stated goals (e.g. Trump's massive tax cuts for the wealthy which Laffer claimed would grow the US economy by 6%).
I'm confident Laffer has Noticed The Skulls, but why should he care? Advice that destroys the US economy but further enriches those who already have more than enough worked out great for Laffer.
If you felt despair when you read the metaphor of totalitarianism and just want to get away from languages that want to grind an axe on you, and Lisp is not your cup of tea, then Raku and Perl welcome you.
This is not about some "technical" impossibility - language still have an escape hatch like unsafe and "non-idiomatic" ways of coding.
But to focus on the technical and ignore programmer experience ("feelz") in that area, is like equating Idris and Forth because "they're all turing complete and it all ends up as binary code anyway".
Whereas working with the borrow checker, or a Haskell-like type system, changes the experience and the method of programming itself, forces certain patterns, and so on.
Using them also changes the experience, and there are common patterns to fake using them.
The thing about "private understandings" is that they're just another way of saying "we expect ceaseless, flawless vigilance" (as one writer put it), not to mention "flawless communication and training."
Languages which impose weird restrictions tend to do so because it allows them to offer (nearly) ironclad guarantees about something else.
There are certain programming idioms that work brilliantly in Haskell. Those same idioms would be utterly miserable in the presence of unrestricted mutation.
Or to take the author's other example, Rust forbids shared mutable state, and it keeps careful track of ownership. But I can freely use native CPU threads without worrying about anything worse than a deadlock, and I can bang directly on raw bytes without worrying about anything worse than a controlled runtime failure. And this remains true even if team communication occasionally fails or if someone makes a mistake.
Sometimes I want to rule out entire classes of potentially dangerous mistakes, and not just have a "private understanding" that nobody will ever make certain mistakes.
As always, it's a matter of using the right tool for the job. If you need to write a high-performance, heavily-threaded network server that parses malicious binary data, Rust is a great tool because of those restrictions. If you need to do highly exploratory programming and invent new idioms to talk about your problem domain, Common Lisp is awesome. And if you need to build libraries where everything has a rigid mathematical structure, Haskell is a great tool.
In my experience, Commony Lisp is a deeply opinionated language, and its most dramatic opinion is that "your code and your data should have identical representations and structures." And for the right problem, that restriction is extremely powerful.