News for hackers. Stuff that matters.
Re:Running Lisp in Production (2020) by reikonomusha on Wednesday February 02, 2022 @01:44AM (#30174557)
We've also been using Lisp in production. It has gone wonderfully, and has allowed some of the most simultaneously maintainable, resilient, and efficient code I've seen in my career. It's also attractive differentiator to some programmers who don't want to drone away writing bulk-Java day-in and day-out.

A key to running Lisp in production is to step away from thinking of it as a lone-wolf bionic-suit hacker language, and to invest in developing a style guide, code review guidelines, CI/CD, and a culture of training new programmers to write Lisp. I've found that you typically don't have to hire Lisp specialists. As long as you find good programmers, they pick up Lisp just fine in just a few short weeks.

The sibling comments are right though: Lisp's reputation is unfortunately affected by so, so many myths that it actually stunts the ecosystem in certain ways. The most rampant myth these days is that (somehow) macros automatically imply doom of a codebase after a certain size or contributor count. Classic myths from decades ago, like "Lisp is interpreted and slow", still echo from time to time too. Lisp's extensive, half-century history is a strength but also clearly a (social) weakness.

A lot of myths come from good-intentioned speculation of the supposed consequences of certain features or aspects of Lisp, and not from actual experience using it.

Re:Running Lisp in Production (2020) by dreamcompiler on Wednesday February 02, 2022 @12:03PM (#30180320)
This has been exactly my experience too. It's easy to bring good programmers up to speed in Lisp. It's sometimes harder to bring existing Lisp programmers onto a team. Let me explain: Some Lisp programmers (not the majority, but some) learned it on their own and it became their personal hobby-project language because it's so powerful. When those programmers join a team they bring bad habits with them. Bad habits like using lists for every data structure, overusing reader macros, not using packages properly, writing their own functions for things covered by well-tested cross-platform quicklisp libraries, etc.

Programming in Lisp is very different than software engineering in Lisp. Of course that's true in any language, but Lisp has a lot more "shoot your team in the foot" possibilities than C, and you absolutely need to learn to avoid those patterns or formally communicate your use of them in order to make Lisp effective on a software engineering team. Rather than depending on the language itself to enforce good software engineering discipline, with Lisp you must create out-of-band agreements with your team about how to write good software. This is not a bad thing; it's something all SWE teams should do anyway. But Lisp is especially intolerant of neglecting this aspect of SWE.

Re:Running Lisp in Production (2020) by xedrac on Wednesday February 02, 2022 @12:41PM (#30180990)
> Lisp has a lot more "shoot your team in the foot" possibilities than C, and you absolutely need to learn to avoid those patterns or formally communicate your use of them in order to make Lisp effective on a software engineering team.

I love Lisp for its adventurous and fun nature, but the above statement is a huge negative for real engineering work on a team. Any time you can rely on your tools or language prevent these types of issues, that's a win. This became especially evident to me when my team started using Rust. Code reviews are now almost purely about logic and design.

Re:Running Lisp in Production (2020) by reikonomusha on Wednesday February 02, 2022 @12:53PM (#30181175)
You don't shoot yourself in the foot in the same way one does in C. With carelessness, you more... suffocate yourself with your own towers of (probably useless) abstractions. It's like having a power drill when all you knew was a screw driver... and then drilling holes and driving screws every chance you get without regard to a larger structure.

Unlike C, the powerful (and safe) language features are a boon to software engineering projects, as long as you're doing was dreamcompiler said: software engineering and not "just programming".

I disagree with the thesis that the more restrictive one is, the better it is for software engineering. We've empirically disproved this time and time again with every supposedly "simple" language evolving way, way past its original design goals. Python, Go, and even Scheme are great examples.

Re:Running Lisp in Production (2020) by dreamcompiler on Wednesday February 02, 2022 @04:26PM (#30184272)
That's a great way to put it. With C you can build a car. With Lisp you can build a better car but it's also very tempting to build a car factory, and a motor factory to supply that, and an iron ore mine to supply the motor factory, and just for grins let's go ahead and reinvent physics in a way that suits our purposes and hey, faster-than-light spaceships are probably a good idea too so let's build those, and pretty soon you've completely forgotten about the car.

Lisp: With great power comes great responsibility.

Re:Running Lisp in Production (2020) by xedrac on Thursday February 03, 2022 @02:26PM (#30197435)
I am not trying to extol the virtues of C. I find C a low bar to improve upon, and I'd prefer Common Lisp over it in many cases.

What I am talking about is having to rely on people to do the "right" things when your toolset could be enforcing many of those things. We've empirically shown time and time again that people are poor at consistently avoiding pitfalls. This is especially true as the size and complexity of a project grows.

Re:Running Lisp in Production (2020) by ScottBurson on Wednesday February 02, 2022 @05:04AM (#30175692)
> Classic myths from decades ago, like "Lisp is interpreted and slow", still echo from time to time too.

Pretty funny, considering how much faster CL is than, say, Python -- never mind Ruby.

Re:Running Lisp in Production (2020) by Mikeb85 on Wednesday February 02, 2022 @06:40AM (#30176267)
> Python -- never mind Ruby

Lol, with the new JIT compilers and TruffleRuby, Ruby is blowing Python out of the water performance-wise. TruffleRuby approaching Nodejs speed for some tasks.

But yes, CL is fast. Faster than any other dynamic language, approaching Java speed (and Java is shockingly quick considering it has a GC).

Re:Running Lisp in Production (2020) by DerArzt on Wednesday February 02, 2022 @09:18AM (#30177857)
It's always cool when we can achieve a speed up in something, but I fear that we may have reached the point of Ruby becoming mostly irrelevant to the larger market. Python, thanks mainly to it's data science libraries, has taken it's place as the go to non-JavaScript scripting language. Unless Ruby comes up with some killer feature like it had with Ruby-on-Rails back in the day, I doubt that the speed up of Ruby will be what revives it for most people.

All of that said, I would love for Ruby to usurp Python from it's thrown!

Re:Running Lisp in Production (2020) by Mikeb85 on Wednesday February 02, 2022 @09:39AM (#30178153)
I mean, no one uses either for its speed. But just funny how reputations linger.
Re:Running Lisp in Production (2020) by runevault on Wednesday February 02, 2022 @12:44PM (#30181032)
Python did a good job offsetting speed issues by rewriting key parts of libraries in C (or maybe C++/fortran in some spots). I don't know if the same effort ever went into key Ruby libraries.
Re:Running Lisp in Production (2020) by samatman on Friday February 11, 2022 @06:34PM (#30308317)
> Faster than any other dynamic language, approaching Java speed

Almost as fast as Java, that's pretty speedy! It's not LuaJIT fast but it ain't half bad!

Lua is also a dynamic language, but who's counting?

Re:Running Lisp in Production (2020) by srean on Wednesday February 02, 2022 @06:34AM (#30176223)
The decline of Lush [0] as a popular data-science language saddens me.

One could use it as a dynamically typed interactive language with nice support for nd array manipulation, excellent support for calling C functions. Once satisfied with correctness of the implementation one could then compile it down to a statically typed C for speed. It ran circles around byte compiled Python in speed. Two of Python's relative advantages in the data-science domain were (i) nicer array manipulation syntax and (ii) one could build the non data-sciency part of the application in Python as well thus avoiding a proliferation of tooling and languages necessary.

Regarding Ruby, even TCL was faster than non JITed Ruby. That was funny because TCL used to be considered the slowest possible runtime (among professionally used runtimes)

[0] http://lush.sourceforge.net/

Re:Running Lisp in Production (2020) by phillc73 on Wednesday February 02, 2022 @12:26PM (#30180708)
There are some other active Lisp efforts in the data science space, such as cl-ana.[1]

[1] https://github.com/ghollisjr/cl-ana

Re:Running Lisp in Production (2020) by hngpt2 on Wednesday February 02, 2022 @05:16AM (#30175772)
I have used C, Java and even Perl extensively for the same "one-off" (like small command-line tools) and "everyday" kind of programming. But it's in those "everyday" kind of programming that I have found most value. The reason is that I often end up doing the same sort of tasks over and over again. As such, I don't want to waste time doing something trivial just because it has to be done so often, but to do something more productive.

For some languages, I could not find much of a reason to use them for such tasks. When in Java I used StringBuilder instead of simply concatenating the values. If I had to add something to the existing string in C, I used memcpy and memset. It's easy enough to understand why. So why would I do the same thing again in Lisp? For the same reasons I did in C?

Re:Running Lisp in Production (2020) by p_l on Wednesday February 02, 2022 @10:21AM (#30178782)
Personally, it used to be the biggest issues in tackling scripting tasks in Lisp in my experience was slow startup (mind you, we're talking about running from 4200 rpm spinning rust). There are people successfully using it for such kind of work, and generally it used to be common in environments with more Lisp support.

These days my biggest issue is that I kind of lost the need for anything more than Bash for one-off tools, and I do it rarely enough that the incremental benefit faces big hurdle of dealing with development-antagonistic environment (mac)

Re:Running Lisp in Production (2020) by zshrdlu on Wednesday February 02, 2022 @02:00AM (#30174637)
As I like to say, Lisp's reputation precedes it more so than any other $TECH.
Re:Running Lisp in Production (2020) by peebz on Wednesday February 02, 2022 @03:03AM (#30174975)
That's really interesting. Can I ask what you use it for and if you've come across any limitations or problems using Lisp?
Re:Running Lisp in Production (2020) by reikonomusha on Wednesday February 02, 2022 @03:09AM (#30175006)
For building optimizing compilers for quantum computers.

The biggest hurdle has been difficulties with deployment. Open-source Lisp implementations mostly[1] only let you build an executable, and not things like a C ABI-compatible shared library. Commercial Lisp implementations are a lot better in this regard.

Even if your OK with building an executable, there are limited options in terms of "enterprise integration", especially in the arena of databases and RPC. Google only very recently released support for gRPC [2], for example, and it's not a first-class citizen in Google's gRPC offering. Most hobbyist Lisp hackers don't exactly care to build (and more importantly maintain) a bunch of open source stuff for enterprise use.

[1] There is a Lisp implementation called "ECL" that compiles Lisp to C, but it's lacking in many other dimensions.

[2] https://github.com/qitab/grpc

Re:Running Lisp in Production (2020) by yvdriess on Wednesday February 02, 2022 @04:48AM (#30175576)
Cool! I used CL for the same purposes in my PhD almost a decade ago. Interesting to see that others independently have been gravitating towards CL.

I had a top-level pattern language with a dataflow-like visual representation, using the LispWorks UI lib to make the editor. That pattern language compiled down to a Measurement Calculus representation (one-way QC, so not the usual circuit model) in s-expr form, which I could both execute with my QVM and do optimizing transformations on. I had a CL QVM was initially CL, but it was quickly replaced with a C + OpenMP version using the same s-expr input.

Re:Running Lisp in Production (2020) by reikonomusha on Wednesday February 02, 2022 @05:05AM (#30175695)
Parallelization and SIMD are pretty easy for numerical code in CL now; I don't think this was the case 10 years ago. LPARALLEL is a nice small library for core-parallelism, and CL-MPI is good for MPI/cluster parallelism. This QVM [1], a pure/density-state simulator, can use either.

[1] MPI version: https://github.com/quil-lang/qvm/tree/master/dqvm

Re:Running Lisp in Production (2020) by agumonkey on Wednesday February 02, 2022 @03:13AM (#30175034)
Are you using SBCL ? also i'm curious what CI/CD platform you used, or maybe you rolled your own ?
Re:Running Lisp in Production (2020) by reikonomusha on Wednesday February 02, 2022 @03:19AM (#30175076)
SBCL, making use of GitLab, Quicklisp dists, Docker, Conda, and Artifactory.
Re:Running Lisp in Production (2020) by agumonkey on Wednesday February 02, 2022 @07:22AM (#30176551)
Oh I wish I could see your lab operate :)
Re:Running Lisp in Production (2020) by vbezhenar on Wednesday February 02, 2022 @09:02AM (#30177621)
How do you convince good programmers to spend their life writing lisp? It's very hard to find lisp work after your company, so it kind of wasted years.
Re:Running Lisp in Production (2020) by reikonomusha on Wednesday February 02, 2022 @01:12PM (#30181479)
The same way I convince them to learn an employer's large code base that will be useless to them after they leave too: by paying them.

That's a real, non-facetious answer. Another facet to the answer is that many people see value in learning a different language to help give them perspective on the existing ones they already know.

I generally don't consider it a net loss to ever learn a new language.

Re:Running Lisp in Production (2020) by Mikeb85 on Wednesday February 02, 2022 @09:08AM (#30177701)
Programming is more than learning languages...
Re:Running Lisp in Production (2020) by oblio on Wednesday February 02, 2022 @09:23AM (#30177918)
For many journeymen developers, the choice of programming language is important. There are many, many companies out there that do look for people with N years of experience with language X and not every developer out there is in a place or at a skill level where they can afford to be picky (the average HN developer isn't representative of this population).
Re:Running Lisp in Production (2020) by Mikeb85 on Wednesday February 02, 2022 @09:24AM (#30177938)
Ah yes, the 'ol "need 10 years of experience in a 5 year old technology"...
Re:Running Lisp in Production (2020) by pmarreck on Wednesday February 02, 2022 @10:16AM (#30178671)
imagine telling an electrician what tools they need to use