When someone says they want to work in Perl6/Raku, they likely have vastly different problems than someone using Erlang/Elixer. There is overlap of course (both general purpose), but I can't imagine using Erlang for scripting, while Raku is first class here.

I can't imagine scripting in Erlang. Or rather, I've seen it done and it wouldn't be my first choice.

Elixir, on the other hand, has many nice features for scripting, such as better string handling, process pipe-lining, and an excellent set of first-class generic data structures (beyond just map and list). It also happens to have Erlang's fantastic concurrency model.

As a testament to this statement I wrote a Google maps API scraper to get a list of driving distances from a pair of csv tables - shortname + address and shortnames to and from, in about 1 hour, and the whole thing clocked in at 90 lines of code including about 40 lines of comments.

That seems about right for a competent Python coder. It or Powershell or Linux command line tools would have very little code and be very easy to whip something up. Perl would be great here as well or honestly any dynamic language with great library support.

Elixir still has to borrow arrays from Erlang and it's a pretty ugly implementation at that so no - I would not agree that Elixir has an excellent set of first-class generic data structures.

Neither Elixir or Erlang have real arrays, unless you want to count tuples. That said, I don’t see how that correlates to scripting, the data structures that are first class in Elixir are very ergonomic and certainly flexible enough to tackle pretty much any task you’d use a scripting language for.

I guess it depends on what you're comfortable in? I myself have tons of Elixir scripts that I use to automate much of my life. It's replaced ruby as my go-to for one-off scripts.

I didn't know that Elixir had such a low-threshold for one-off programs.

Have you tried Crystal? Both are on my list of programming languages to look into next, together with Elm, Reason and Zig.

Oh man I just love and adore Crystal. I am a long time rubyist and I have to say they just nailed it with Crystal. I just wish I had more opportunity to use it professionally!

It must be time to take my Ruby skills and go and learn this. Is it really as easy as it sounds like it should be for a lazy, dynamic-typing sort of Ruby dev to pick up?

If you have never worked with a strongly typed language before there might be some getting used to that, but any pain is well worth it. Catching all your mistakes at compile time and not at runtime is so amazing. Oh and it’s literally 100x faster than Ruby in some cases.

> Catching all your mistakes at compile time and not at runtime is so amazing

It would be, but IME Crystal, or statically typed programming languages that don't have a type system at least as powerful as Haskells in general, don't get anywhere remotely close to that, and the overhead added to get what they do give is of dubious net benefit outside of large projects.

And even with Haskell, while you can express a much richer set of constraints in types, there's also more space for bugs in the type level programming.

I don't get it. How many types do I need, before the kind of data I am talking about becomes better represented by an object comprised of smaller pieces?

Why would I want to spend my time "type level programming"?

Maybe I don't understand what's meant by a type. Is it on the level of string/double/int, or is it expected to cover more (string with a well formatted ISO 8601 datetime, etc)? What is accomplished that couldn't be done in vanilla Ruby with an object representing a more complex datatype with its own getters/setters to cover validation?

Every mistake? Wow. I rarely make that kind of mistake, I usually screw up the logic.

Crystal is wonderful for one-off programs in my experience.

Elixir had a long startup time for one-off scripts in my experience, but Elixir’s syntax is still one of my favorites for scripting.

Is a 100ms startup time really a big deal for something that is either going to be manually triggered or cronjobbed and expected to take on the order of minutes?

Those sorts of startup times discourage writing command-line tools (e.g. composable text processing things one might join with pipes).

Ruby scripting has occupied the sysadmin space for over a decade yet on my 2013 Macbook Pro the latest Ruby 2.6.5 takes around 250ms to start. That's a mere 1/4 of a second. Sure, Python, Perl and PHP have shorter startup times but surely this is a non-problem?

No, things like sort, grep, uniq, cut, wc, cat, awk, sed, tr, etc etc. You wouldn't want them to take 1/4 second to start up.

Crystal's startup time is not great though. 700ms vs 100ms for ruby.

Compared with

   time crystal eval <<<"puts 1"
   time ruby <<<"puts 1"

You know that you are timing the compiler and not the compiled program here?

Rather try "echo 'puts 1' > puts.cr && crystal build puts.cr && time ./puts"

BTW, you can dramatically decrease the cold startup times of ruby if you call it with the options '--disable-gems --disable-rubyopt' if that's an option to you. For many scripts it certainly is.

But even on a 8 years old machine with spinning rust, after the ruby stuff is cached in the OS filesystem cache, my startup time are smaller than 10ms.

Elixir/Erlang isn't a general purpose language. You pointed-out one reason why - scripting. Computation is another area where the language is deficient. I'd go so far as to say Elixir's failure to challenge mainstream languages for mindshare is down to to it occupying a niche - lightweight concurrency and distributed systems.

Their point was that elixir is a good language for scripting. I'm not sure if this is true but I don't see any reason it couldn't be. I don't think "distributed systems" are niche... I think every web application I've worked on fits that description, which is basically my whole career at all different kinds of companies.

I'm trying to understand, what kind of scripting requires first-class concurrency that isn't fulfilled by say Python?

For concurrency I think Raku has slightly more developed async support than Python, but the bigger advantage, I think, is in parallelism where, aside from the CPython GIL limiting practical parallelism in the main implementation (which is a big deal), Python as a language lacks the parallel iterables produced by the hyper (parallel but constrained to return in order) and race (parallel and not constrained in order) methods on the Iterable role in Raku, or anything like Raku’s hyperoperators that are semantically concurrent and parallelizable at the discretion of the optimizer. (Come to think of it, while also parallel, all those are also high-level concurrency constructs that Python lacks equivalents to.)

Python as a language can support parallelism via threads, and CPython as an implementation can via multiprocessing, but those are both very low level abstractions; Raku has higher level abstractions which allow much more succinct expression of parallel operations.

Sounds like multiprocessing.Pool.imap and imap_unordered? When dealing with io you also have async/await equivalent iterators like asyncio.as_completed().

Raku has `await` but it doesn't need `async`.

    sub foo ( Int \n where 1..100 ) {
      #  ___ start is a prefix operator that creates a Promise
      # V
      start { sleep n.rand; (1..n).pick }
    }

    my $result = foo(10);

    say $result.status; # Planned

    say await $result; # 6 (after a delay)



    # this is nearly identical to foo above
    sub bar ( Int \n where 1..100 ) {
      Promise.in( n.rand ).then({ (1..n).pick })
    }

(Note that `start` does not need a block unless you are grouping more than one statement as I have above.)

There are combinators like `Promise.allof` and `Promise.anyof`.

You usually don't need to use `Promise.allof`, because you can use `await` on a list.

    my @promises = Promise.in(1.rand) xx 10;

    my @results = await @promises;

(Note that the left side of `xx` is _thunky_ so there are ten different Promises with ten different random wait times.)

---

You should see the `react`/`supply` and `whenever` feature.

I am not that familiar with Python, but the GIL has long prevented any real in-process concurrency. Perl has concurrency but it's complex, heavy, and poorly supported. Raku's approach to this is built to avoid all these problems (like Elixir).

I agree the GIL is a problem but it's only an issue for CPU-bound problems. Is there really an important amount of CPU-bound work that is written in a scripting language? If it's CPU-bound, wouldn't you want to use something lower level?

If it's entirely CPU bound, you can use multiprocessing to negate most of the GIL issues, and transparently send inputs/outputs between the parent and child processes. If it's I/O bound, then AsyncIO is a great way to express asynchronous workflows. If it's a combination of both I/O and CPU bound workloads, there are ways to mix multiprocessing and AsyncIO to better saturate your cores without losing the simplicity or readability of Python: https://github.com/jreese/aiomultiprocess

Very true, I had not even thought about the multiprocessing package; it's sometimes not as convenient as multithreading but it'll get those other cores working.

Indeed and as a Perl developer I make use of XS/external libraries, cooperative multitasking (event loops/promises), and forking to cover these use cases. It doesn't preclude wanting the additional option to take advantage of threads in a useful way, since they do exist.

It's not just CPU bound problems, handling multiple overlapping i/o operations is more trouble than it ought to be.

Can you expand a bit on that? I'm not familiar with the issue you're describing.

Why should first-class concurrency needs be required to script in Elixir? This question seems to imply that Python is somehow a default language and special requirements must be needed to justify writing in something else. Elixir is general-use and pleasant to write scripts in so seems reasonable to me for someone to do so if that's their thing.

Ovid2 said Perl6 has a good concurrency model

lliamander replied that Elixir does too and it's worth a look

To that, 7thaccount replied that Perl6 and Elixir fill different niches.

So far, it seems Perl6 fills a niche that requires scripting and first-class concurrency. My question then is: what is this niche that requires very solid concurrency but also scripting. In other words, what does Perl6 have in terms of concurrency that Python does not (given they are both scripting languages)?

A fair question.

I don't know of many instances where scripting and concurrency would be needed in the same application. But if you wanted to use single language both for scripting tasks and applications that require concurrency, then Raku or Elixir would work.

One instance I can actually think of, that would be specific to Erlang/Elixir, is if you have a long running service that you sometimes run batch jobs against (like importing data from a CSV). An Elixir script can hook directly into an Elixir/Erlang service and interact with it using message passing. It's a heck of a lot simpler than RMI, and avoids the need to expose new HTTP endpoints specifically for such tasks.

Is that like the relationship between C# and PowerShell?

I think so, at least in some ways. I've shipped a project using PowerShell to script Windows Server/Hyper-V, and it was a pretty pleasant experience. Having a scripting language that not only does typical scripting stuff (wrangling text, etc.) and understands your application's objects is excellent.

Some differences:

* You can actually write your whole application in Elixir, whereas I could not see doing that with PowerShell

* In Erlang/Elixir, instead of objects you have processes. Think of your application as a microservice architecture on steroids, using the Erlang RPC protocol as a means for inter-process communication.

Because each process just sends and receives messages, your script can just send messages to any of the processes that make up your application, as if they were your own service. All you have to do is connect to the remote application using the Erlang Distribution Protocol (to connect different runtimes over the network).

I heard so much about the actor model, I should really try it in its intended glory one day.

It's a trip, and it took me a little while to wrap my head around. However, I now much prefer it to working with other concurrency abstractions (such as callbacks).

Jonathan Worthington (Rakudo Compiler Architect). Keynote: Perl 6 Concurrency

https://www.youtube.com/watch?v=hGyzsviI48M

Does Python 3 have any operators that transform ordinary method calls into concurrent method calls? Perl 6/Raku does.

More to the point, does Python 3 have operators that transform sequential operations into operations that are both concurrent and parallelizable, and, in the case of iteration, provide control of parallelism parameters and whether results are constrained to be in-order or not.

To which the answer is “not only does Python not have them, but with the GIL CPython couldn't do much with them even if the language had them.”

Is that a bit like Go's `go` keyword?

edit: that is, as far as I can tell, after a quick Google, it's not too different from Python's Thread object.

The `start` statement prefix operator is a bit like Go's `go` keyword.

(In that when I translate Go into Raku, I usually replace `go` with `start`.)

    my Channel \input .= new;
    my Channel \output .= new;

    start for input -> \message { output.send(message) }

    start for output -> \message { say message }

    input.send(5);

But what he was really talking about is something more like the following.

    sub foo () {
      sleep 10;
      42
    }

    #                     ___
    #                    V   V
    my $delayed-result = start foo();

    say 'Hello World'; # Hello World

    say $delayed-result.status; # Planned

    say await $delayed-result; # 42

It is, python Threads are system threads.

Yes, async/await. https://docs.python.org/3/library/asyncio-task.html#awaitabl...

Which is a similar response to the comments asking "why Perl over Python?" I ask, why not (both)?

Especially since you can use the Inline::Python module.

I'm embarrased to say I write scripts in node :P I used to write scripts in python but I've been writing so much JS that it's just easier for me in node. Plus, node defaults to locally installed deps so I don't have to deal with virutal environments.

Why do you think this is embarrassing? Javascript is great for small scripts. Largr codebases then sure

Well just from experience untangling async calls in python is a nightmare and sometimes hard to reason about. The red/blue function problem is real. Meanwhile dispatching concurrent long-running scripting tasks is basically trivial in elixir (Enum.map over Task.async, then Enum.map over Task.await)

I'm guessing because Python's concurrency relies on the Global Interpreter Lock. Although I think concurrent.futures might address that. Haven't worked with python concurrency libraries in a bit.

as I posted above:

I agree the GIL is a problem but it's only an issue for CPU-bound problems. Is there really an important amount of CPU-bound work that is written in a scripting language? If it's CPU-bound, wouldn't you want to use something lower level?

There is machine learning, which usually calls into numpy or other c extensions With a lot of the data preparation done in python

tldr; Using a scripting language that allows for native threads or has a strong concurrency model builtin to the core would be beneficial for any CPU bound scripting task...

-----

Python's concurrency model is good for waiting on network or disk I/O because of its GIL (Global Interpreter Lock): https://realpython.com/python-gil/#the-impact-on-multi-threa...

If your program is CPU bound the GIL will slow you down. I'm sure since the python ecosystem is quite vast there are ways around the GIL... but then you have to worry about every library you use not supporting "real" multi-threading, or (b)locking for no reason and shitting all over your efforts.

As I've posted above, I'm a bit confused by CPU-bound work being processed in a scripting language. If you're planning on doing intense CPU-bound work, maybe use a lower-level language? I'm not saying abandon Python: you can extend Python with C or just use IPC to transfer data between a Python front-end and a computation back-end.

I have a different perspective.

When I have a bit of Raku code that is too slow I complain (send a bug report) and then someone else figures out why the JIT isn't doing a better job and fixes it.

Then bingo-bango it ends up even faster than using NativeCall to call into the C code.

Of course there may be a delay before someone gets around to figuring it out; so in the meantime, NativeCall is actually very easy to use.

---

I would like to note that someone wrote Perl6/Raku code and the equivalent C/C++ code. They reported that the Perl6/Raku code was faster. (It was before Raku was brought up as an alias/name.)

I'm fairly sure that the reason the C/C++ code was slower is that it copied strings around rather than using pointers and indexes into other strings like MoarVM does.

At one time it was an obvious dichotomy that you would not use a scripting language for CPU bound work, but these days it is a much more blurry line. Partly because modern efficient languages are becoming ergonomic enough to work well as scripting languages while still giving you very good performance.

I actually love doing CPU bound work in Groovy which is usually described as a scripting language. But it gets converted straight to java byte code which is JIT'd and ends up as machine code. It only takes a few static typed hints here and there and it runs in the same league as C++ implementations. And it gets Java's excellent concurrency support for free.

I totally you feel you, I guess I thought your question was substantially more surface level than it was. My apologies.

I'm personally with you. I also don't tend to think object boxing is really the performance bottleneck for most applications, and if/when it is, likely the other requirements should've already ruled out using one (a scripting language).

It's like writing Nifs for Elixir, yeah sure you _can_, they have their purpose, but you could also just write another application to do that one thing and like you said, use IPC.

So in summary, we agree with each other, here's to:

the right tool for the job!

> the right tool for the job!

Hear, hear!

For that matter, Erlang has been around for like almost 30 years and has had multicore support since 2007, and has always been dynamic.

Erlang is great, and I miss working with it, but I'd never want to write much in the way of 'quick scripts' with it. Something like Ruby feels much more productive for that.

Heh, escript isn't so bad once you get used to it.

Clojure would be a great language for small-ish scripts if it weren't the dog-slow startup times, and has excellent concurrency support.

I hear that GraalVM might fix that but I sadly haven't had a chance to play with that yet.

Even if GraalVM fixes startup times aren't JVM languages just too long-winded for scripting? Perl, Python and Ruby dominate the scripting world for a reason - standard libraries for file, dir and pathname manipulation written in a concise language. Scripting is a style of coding, not just a means to an end. It is here that dynamic languages excel. Clojure is the leanest of the JVMs but doesn't it still rely on Java for file, dir and pathname manipulation?

> Even if GraalVM fixes startup times aren't JVM languages just too long-winded for scripting? Perl, Python and Ruby dominate the scripting world for a reason

Ruby is a JVM language, in that JRuby is a very complete competitive, current, and widely-used-in-production Ruby implementation.

By JVM languages I really meant those designed originally for the JVM, ie. Java, Clojure, Scala, Kotlin & Groovy.

Even so, Clojure and Groovy aren't particularly long-winded, nor is Scala despite static typing.

You should definitely educate yourself about Groovy. It's my favorite scripting language, particularly because it enables a true scripting style, but actually works well as a full application development language too (though I would always want a backbone of static typed code).

It relies on the implementation of its runtime, JVM, .net, JavaScript, BeamVM etc but uses it's own language for those operations, slurp/barf etc

So as a developer it's all the same until you start leveraging the features of your particular runtime but this does make code sharing viable

There are JVM versions of Python, Ruby and Tcl, so it's not really about the VM itself.

Ya, GraalVM fixes it. But it requires a compile step, so it's not as nice for scripts.

My favourite for scripting right now is Joker: https://github.com/candid82/joker

Joker is a Clojure dialect which is interpreted, thus it starts super fast (but runs slower, but fast enough for scripts). Its design is to be batteries included for all things scripting. So it's just a self contained executable with everything you need for scripting. It's implemented in Go.

I use it wherever I would have used bash or powershell prior.

There is also Babashka: https://github.com/borkdude/babashka which is a similar idea, it's an interpreted dialect of Clojure with fast start times designed for scripting. The difference with Joker is that it is newer and more experimental, and it is implemented in JVM Clojure, compiled with GraalVM and has no Windows support for now.

Anyways, I really recommend the use of Joker. Its awesome for scripting. I just put its binary in all my bin folders and script with it. It's great.

GraalVM does in fact fix that. There's a (new-ish) project called babashka that lets you do some basic scripting for example: https://github.com/borkdude/babashka

Oh certainly. I've done production work with Erlang, and it's great for building systems. I also love the Prolog syntax (I've also since done some side projects in Prolog for great fun).

Why I suggested Elixir is because it has all the strengths of Erlang but is also a better scripting language, and would compete better with Raku on a more equal footing in that respect.

In fact Elixir and Erlang are the only reasonably known dynamic languages that differentiate on concurrency. The rest either do higher order event loop concurrency, which is ok, but not a differentiator, or do worse, shared memory multithreading (threads, coroutines with synchronous channels, mutexes, etc.)

I don't really know what "differentiate" means, but Clojure has support for CSP-channel style with core.async, shared-memory with transactions using atoms/refs, and actor-ish style with the agents.

> I don't really know what "differentiate" means [...]

Probably “set themselves apart from others”.