bradcray

I disagree with the characterization that Chapel's parallelization features copied OpenMP without improving upon it:

* Chapel's support for task parallelism predates OpenMP's (~2004 vs. ~2007, where Wikipedia cites Chapel's tasks as being inspiration for OpenMP's, along with Cilk and X10). Chapel's tasks are also arguably more general-purpose (akin to threads) in terms of their ability to synchronize, support data-driven producer/consumer patterns, etc.

* Chapel's forall loops are similar to OpenMP's loop-based parallelization pragmas, though OpenMP wasn't a source of inspiration in their design. Where OpenMP pragmas select from a menu of parallelization strategies baked into the specification and implementation, Chapel's forall loops invoke user-defined parallel iterators that permit abstracting a particular parallel pattern (say, multidimensional tiled iteration or tree traversal) into a named subroutine. These iterators can optionally be made methods of data structures and/or placed within libraries, and can be re-used across a program or multiple programs. One such library, DynamicIters, was community-contributed and specifically inspired by OpenMP's dynamic and guided scheduling strategies.

* Chapel supports parallel zippered iteration, in which two or more data structures and/or parallel iterators can be traversed in a coordinated manner.

* Chapel's parallelism can span multiple compute nodes via its shared namespace, which obviates the need for explicit communication; whereas OpenMP is limited to a single compute node or process unless mixed with MPI, SHMEM, or the like (and even then, OpenMP doesn't gain a cross-node view of parallel computation).

* In Chapel, parallelism can be expressed implicitly, for example, by passing an array argument to a subroutine or operator that is expecting a scalar (e.g., `var B = sin(A);` or `var C = A + B;`).

Good point on my alluding to demerits of Fortran, C, and C++ without stating them, and thanks for clarifying your criticism. Using the four factors that I focused on as attractive features in new languages:

Productivity: For me, while Fortran has some nice features for HPC (multidimensional arrays), lots about its design feels very old-fashioned to my (not particularly young) eyes. C and C++ are more "my generation" of programming language, so are familiar and comfortable, yet they still seem verbose, convoluted, and less readable (more symbolically oriented) as compared to Python, Julia, or Swift, which are more what I'm looking for in terms of productivity these days. Of the three, C++ has clearly made the biggest strides in recent years to improve productivity, with some successes in my opinion, though I've also had a hard time keeping up with all the changes.

Safety: I consider C and C++ to be fairly unsafe languages compared to more modern alternatives. I don't have enough experience with Fortran to have a particularly informed opinion, but feel as though I've been aware of patterns in the past that have felt unsafe. Here again, I think using modern C++ in a certain style (e.g., smart pointers) probably makes nice strides w.r.t. safety, but I'd still consider there to be a gap between it and Python/Rust (as does my colleague in this post: https://chapel-lang.org/blog/posts/memory-safety/)

Portability: Modulo the degree to which various compilers keep up with the latest standards in Fortran and C++, I'd consider all three languages to be quite portable.

Performance: There's no question that these are high-performing languages in the sequential computing setting. In HPC, while Fortran or C++ and MPI are often considered the gold standard, it's a standard that can be beat if your language maps more natively to the network's capabilities, or knows how to optimize for distributed memory computing rather than relying on the programmer to do it themselves.

With respect to the "10 myths" series, while the focus of the series was about combatting prevalent negative attitudes about new languages in the HPC community, I think there's a lot of content along the way that rationalizes the value of creating new languages in my rebuttals. That said, I fully realize that it's a long read, particularly in its updated "Redux" form.

Thanks again for clarifying your previous point.

I'd say that the opposite is more often the reality, which is why HPC systems tend to have high-bandwidth, low-latency networks.

The evolution of C, C++, and Fortran is touched on in a sidebar, although admittedly very briefly:

> Champions of Fortran, C++, MPI, or other entries on this list could argue that…

Author here: I didn't go into more detail on this than https://chapel-lang.org/blog/posts/30years/#maybe-hpc-doesnt... because I felt like the article was long enough already and that I'd recently covered that topic in detail in this series https://chapel-lang.org/blog/series/10-myths-about-scalable-... summarized here https://chapel-lang.org/blog/posts/10myths-part8/#summary

@yubblegum: I'm afraid we don't have an update on support for Apple GPUs since last year's comment. While it comes up from time-to-time, nobody has opened an issue for it yet (please feel encouraged to!), and it isn't something we've had the chance to prioritize, where a lot of our recent work has focused on improving tooling support and addressing user requests.

I'll take your feedback about simplifying GPU-based installs back to our team, and have noted it on this thematically related issue: https://github.com/chapel-lang/chapel/issues/25187#issuecomm...

@yubblegum: I'm unfairly biased towards Chapel (positively), so won't try to characterize HN's opinion on it. But I did want to note that while Chapel's original and main reason for being is HPC, now that everyone lives in a parallel-computing world, users also benefits from using Chapel in desktop environments where they want to do multicore and/or GPU programming. One such example is covered in this interview with an atmospheric science researcher for whom it has replaced Python as his go-to desktop language: https://chapel-lang.org/blog/posts/7qs-dias/

The ~10-minute video for this talk is here, if anyone's interested in the narrative behind the slides: https://www.youtube.com/watch?v=U8KM8wv32js

We agree that the placement of data is important for HPC programmers to control. Locales are the means of controlling such placement in Chapel, whether directly (as in this article’s simple examples) of via abstractions like distributed arrays (whose implementations rely on locales).

Once the data is created, computations can be executed with affinity to a specific variable in a data-driven manner using patterns like `on myVar do foo(myVar, anotherVar)`. Alternatively, an abstraction can abstract such details away from a user's concern and control the affinity within its implementation, as the parallel iterator implementing `forall elem in MyDistributedArray` does.

That doesn't seem extreme to me, as I generally feel similarly. If you (or other readers) are genuinely interested in using Chapel with Metal, please open an issue on our GitHub repository capturing your request, as that would be valuable to us.

Just to make sure it didn’t get lost, note that it is possible to develop GPU code in Chapel on a MacBook using the cpu-as-device mode Engin mentions above, and then deploy it on NVIDIA GPUs on production systems by recompiling. This is how I develop/debug GPU computations in Chapel.

These are great questions, and ones we’re very curious about as well. I don’t believe that our current Chapel team has much experience programming NNs and LLMs, having focused on other areas. That said, I’m also not aware of any intrinsic barriers to implementing such algorithms in a portable way within Chapel, potentially calling out to vendor-optimized implementations when available and appropriate.

If you, or others, would be interested in exploring this topic, we’d be very interested in either partnering with you or supporting your efforts.

(Also see Engin's response about programming tensor cores for some thematically related thoughts: https://news.ycombinator.com/item?id=39020703 )

Chapel was designed for the high performance computing community where programmers often want full control over mapping their computations to their hardware resources without needing to rely on techniques like virtualization or runtime load balancing, which can obscure key details. That said, higher-level abstractions can be (and have been) written in Chapel to insulate many computations from these system-level details, such as distributed arrays and iterators. Users of these higher-level features need not worry about the details of the underlying locales. We refer to this as Chapel's support for multiresolution programming.

That said, other communities may obviously prefer different approaches due to differing needs and constraints.

Sorry for what now appears to be a double-post. Engin had just registered for HN, hadn't seen his reply going through, so asked me to relay it.

Re-reading this Q+A this morning, I also wanted to clarify one thing, which is that when a 'foreach' or 'forall' does end up being executed on the CPU, that doesn't mean it has been serialized. 'foreach' loops on the CPU are candidates for vectorization while 'forall' loops typically result in multicore task-parallelism with each task also being a candidate for vectorization.

@ColonelPhantom: Thanks very much for your questions. The following are answers I'm relaying from Engin Kayraklioglu, who heads up the Chapel GPU effort:

Re Intel support: That's definitely in our plans. However, there are also many other areas where we are actively working on to add more features, fix bugs, and improve performance. When prioritizing, we typically make decisions based on what our current and potential users might need in the language. Frankly, we are not seeing a big push for Intel GPU support so far. So, currently it is not near the top of our priorities. If you (or other readers) have any input on that matter where lack of Intel support might be a blocker for testing Chapel and/or its GPU support out, definitely let us know.

Re implicit serialization: To clarify; the serialization based on order-dependence is not implicit. The users should use a `for` loop if their loop is order-dependent and `foreach` (and `forall`) if their loop is order-independent. In other words, the Chapel compiler doesn't make decisions about order-dependence. In particular, for GPU execution a `for` loop will never turn into a GPU kernel.

There are, however, some cases where a `foreach` does not turn into a kernel. You may be referring to those cases, but that's not related to order-dependence. Some Chapel features cannot execute on a GPU. If your `foreach` loop's body uses any of those features then it will not be launched as a kernel even though `foreach` signals order-independence. Now, a subset of such features that makes an order-independent loop GPU-ineligible are there because we haven't gotten a chance to properly address them, yet. Another subset of such features will remain thwarters for a longer time and maybe forever. For example, your `foreach` loop could be calling an external host function.

I would say Chapel was created less to replace MPI and more to provide a higher-level alternative to it that is amenable to compiler optimization.

Those interested in the intersection between Python, HPC, and data science may want to take a look at Arkouda, which is a Python package for data science at massive scales (TB of memory) at interactive rates (seconds), powered by Chapel:

* https://github.com/Bears-R-Us/arkouda * https://twitter.com/ChapelLanguage/status/168858897773200179...

My answer would be that Chapel supports a partitioned global namespace such that a variable within the lexical scope of a given statement can be referenced whether it is local to that CPU's memory, stored on a remote compute node, or stored within a GPU's memory (say). The compiler and runtime implement the communication on the programmer's behalf and take steps to optimize away unnecessary communication. Other key features include first-class support for creating parallel tasks in high-level ways, including parallel loops.

@cbkeller: Though MPI is dominant in HPC by a very large margin, it's definitely not the only game in town. SHMEM is an MPI alternative with a smaller but very dedicated following. UPC, Fortran 2008, UPC++, and Chapel are all alternatives that support inter-node communication without relying on MPI or explicit library communication calls. Chapel has the additional advantage of not imposing the SPMD programming model on the user and supporting asynchronous dynamic tasking.

It's my understanding that Julia aspires to join this group of languages if it is able to do so, which is why the Petaflops announcement was originally enticing to me, and then became somewhat less so once I learned that it was relying on MPI.

My understanding is that the Julia Petaflops run executed a Julia program per node, communicating via MPI. For some, that's probably obvious/expected for HPC; for others, it might not be considered "pure Julia".

Thanks for the mention, @distortedsignal! Note that CHIUW 2023 is online and free, requiring just a quick Zoom registration from the URL above.

Tomorrow's topics are more about reporting on cool things being done with the Chapel language than helping people get started with it, but should still be reasonably approachable for technical folks who are new to the language.

For those who missed today's sessions (which were more about getting started with the language), we can schedule make-ups on an ad hoc basis as well.

@blueslurpee: I'm definitely biased, but agree that Chapel would be a fun language to explore, particularly if you're interested in looking at integrated task- and data-parallelism within a language or distributed computing with a partitioned global namespace. If you do, feel free to reach out with questions/comments/requests on Stack Overflow, Discourse, GitHub, Gitter, etc. [1]

-Brad

[1]: https://chapel-lang.org/community.html

> Chapel, HPC language mostly sponsored by Intel and HPC

Minor correction: Intel hasn't traditionally been a sponsor of Chapel (though we'd love to see that change). Chapel was pioneered at Cray Inc. and continues on even stronger at HPE after its acquisition of Cray.

-Brad

@mardifoufs: I realize that your point is that the Chapel webpage didn't answer this question clearly / concisely for you and agree that we could and should improve that. The observations in this thread have definitely helped given me insights about how we could position ourselves better for the non-HPC audience (and I already have a longstanding intention to make the code sample on the page more compelling using Asciinema that I need to find the time for).

But to answer your specific question here: I'd say that most mainstream languages (e.g., C, C++, C#, Java, Python, Rust, Swift, ...) don't aspire to scalability in the same sense as Chapel and High Performance Computing want it, in the sense of being able to run efficiently on tens of thousands of processors with distributed memory where inter-processor communication and coordination is required. And even when they do aspire to it, it's rarely through the language itself, but through communication libraries, pragmas, and extensions. The result (in my opinion) is rarely as productive, general-purpose, and performant Chapel achieves.

I think it's plausible you would want to use Chapel for every program you write. I definitely want to use it for every program I write, but I'm also biased.

The main disincentive to doing so today is that Chapel is not nearly as broadly adopted or well-supported as the languages you probably do use in practice today. The Chapel team is trying to get it to that point, but it's a modest-sized team taking on large challenges (both technical and social, as this thread indicates). To date, we haven't made a significant effort to draw in a massive/mainstream audience because we know we're not ready for it yet, either in terms of the language's maturity or our ability to support a large group of users. But we hope and intend to get there.

That said, I think we're already achieving the aims in the slide you quote—far more than your easy answer suggests—though there's obviously room for differences of opinion (e.g., what does it really mean to be "as programmable as Python?"). If you're interested in pointers to supporting details, let me know.

Arkouda [1], [2] is a recent and substantial example use of Chapel. It's a Python package that supports a subset of NumPy operations on distributed Terabyte-scale arrays implemented using a server written in Chapel. It currently involves ~10k lines of Chapel code.

Other recent notable examples include using Chapel for Dark Matter simulations [3], tree-based search algorithms [4], and CFD simulations (to appear at CHIUW 2020 [5]).

Your point is taken that the Chapel webpage could be better at pointing to such examples via a "powered by Chapel" section or the like.

[1]: http://www.clsac.org/uploads/5/0/6/3/50633811/2019-reus-arku... [2]: https://chapel-lang.org/presentations/ArkoudaForPuPPy-presen... [3]: https://github.com/mhmerrill/arkouda

[3]: https://github.com/sourceryinstitute/PAW/raw/gh-pages/PAW-AT... [4]: https://link.springer.com/chapter/10.1007%2F978-3-030-22734-... [5]: https://chapel-lang.org/CHIUW2020.html

Chapel performance has improved by leaps and bounds over the past 5+ years, though link [2] above focuses on single-node performance whereas most the team's recent optimization efforts have been focused on improving distributed memory performance and scalability, some indications of which can be seen at [3].

Of course, it's very possible for programmers without Chapel expertise to write and publish bad performance results in Chapel. The Chapel team is very interested in working with such users to help them understand where performance is being lost and to improve things to benefit future users. Most of our recent optimizations have been motivated by user feedback, and in practice we're typically able to help users get their codes to match or beat the performance of conventional approaches with reduced effort. A favorite recent example of mine can be seen in slides 6-16 of Nikhil Padmanabhan's talk at PAW-ATM 2019 [4].

A good way to keep up with Chapel performance improvements beyond whatever gets peppered into our talks is through the release notes that we publish every six months, e.g. [5], [6], [7].

[3]: https://chapel-lang.org/perf/16-node-xc/?configs=gnuugniqthr...

[4]: https://github.com/sourceryinstitute/PAW/raw/gh-pages/PAW-AT...

[5]: https://chapel-lang.org/releaseNotes/1.20/06-perf-opt.pdf [6]: https://chapel-lang.org/releaseNotes/1.19/05-benchmark-opts.... [7]: https://chapel-lang.org/releaseNotes.html

Historically, Chapel had an effort to support mixed instruction sets within a single logical program, but this effort fell by the wayside due to the amount of effort required to maintain it, combined with the fact that it was virtually unused (most HPC programmers are using homogeneous nodes, or at least ones with compatible ISAs).

Chapel can still be used in multi-architecture distributed computing today, but by running a Chapel program per ISA and having them communicate through more traditional means (e.g., ZMQ).

kenko is correct that the 'config' keyword is what indicates that a symbol can receive a command-line override in Chapel. As others have suggested, we followed C's lead in interpreting 'const' as "Cannot be re-assigned after initialization" rather than "Value must be known at compile-time."

To specify values that must be known at compile-time, Chapel uses the 'param' keyword, where 'config param' symbols can be set on the compiler's command-line (as can 'config type' symbols, used to specify type aliases). 'param's and 'type' symbols tend to lead to compile-time specialization, similar to C++ template variables.

That just leaves 'config var' which indicates that a symbol can be set on the execution-time command-line and re-assigned over its lifetime.