I'll add that it's well suited to modern AI inference and several projects exist, e.g.
https://github.com/nlpodyssey/rwkv.f90
https://github.com/certik/fastGPT
https://github.com/rbitr/llm.f90 (disclaimer, mine)
https://github.com/modern-fortran/neural-fortran (disclaimer, I originally created this one)
Caltech has a project attempting to write a new climate model in Julia:
Getting a weather or climate model from zero to production grade requires approximately 100 person-years, or $20M (personal experience). Because of extremely high scientific expertise needed to correctly implement such models, it's more difficult to leverage open source contributions from a broader community, like it is with web or ML frameworks. So most of the development in these projects is done by full-time hires, and to a lesser extent by contributions from early adopters.
The key technical arguments that I hear/read for such transition projects are ease of portability to accelerators (e.g. GPUs), and higher availability of programmers in the workforce.
My intuition is that a $20M budget, if carefully allocated to compiler and hardware accelerator teams, could solve running Fortran on any accelerator in its entirety.
With Fortran's tooling and compiler support for offloading standard Fortran to accelerators slowly but surely improving over time, the rationale for porting to other languages becomes increasingly more questionable and is being revisited.
But regardless of technical arguments, it's a good thing for large models and frameworks to be written and explored in diverse programming languages.
Also, I understand the only Fortran compiler that supports GPU is the one from nvidia which is proprietary. I prefer to rely on open source for a code base that will last at least ten years...
But reading this HN's thread, I understand that Fortran is more alive than I thought. How many new developments are done with Fortran ? I mean, to me, Fortran is a bit like Cobol: it is so entrenched that, obviously, it still have a lot of activity but the momentum is moving towards more modern languages... But, well, that's all guesses an impressions...
Many high performance numerical computing libraries are “using FORTRAN” in the sense that they’re linking binaries compiled from FORTRAN for numerical linear algebra functionality. Cf BLAS (Basic Linear Algebra Subprograms).
But most of the ecosystem other than BLAS doesn’t share this property.
EDIT: Wow, I didn't expect so many informative and reasonable replies. Kinda makes me want to check out Fortran. Thanks y'all!
I generally prefer Julia, as its a more general-purpose language, but there are parts of Fortran I like better than Julia, such as
- Fortran uses static typing and is statically-compiled
- It's a lot easier to write slow Julia code than I'd like, and you generally need to think more to make code fast than you do in Fortran.
However, I think Julia beats Fortran in most everything else. My main gripe with Fortran these days is 1) lack of a decent default package manager (FPM seems great, but most Fortran codes don't use it) and 2) slow evolution due to the conservative standards committee. I don't understand why we still can't have extremely basic generics in 2023, or a simple string type.
In C++ it is possible to define adequate types and operations for good support of multi-dimensional arrays, but this means that a user must choose some additional libraries to get the support that does not exist in the base language, unlike with Fortran.
C++ can be much better than Fortran for scientific computing, but only after a significant initial effort of establishing a well-specified programming style, based on selecting an appropriate non-obsolete subset of C++ and after selecting carefully a set of libraries that provide all the missing features.
For someone whose main job is not programming, using Fortran can be simpler.
m[0::2, 0::2] // Numpy
m(0::2, 0::2) // Fortran [1]
m(seq(0, last, 2), seq(0, last, 2)) // C++ Eigen
My fingers can't type Fortran anymore, so I'm going to use C as an example.
Imagine you have this function:
/* Add entries in arg1 to arg2, put result in result */
void add_vec(int arg1[], int arg2[], int result[], int length) {
for(int i = 0; i < length; i++) {
result[i] = arg1[i] + arg2[i];
}
}
int array[101];
// ...pretend array is initialized...
// for the first 100 elements, set a[i] = a[i] + a[i+1]
add_vec(array, array+1, array);
In Fortran, the compiler knows that the arrays are not aliased, so it can do much more reordering, unrolling, and vectorization to speed this code up.
Is there much going on with Fortran on GPU?
Also, for at least the older versions of it, its a pretty small language. If you have a grad student who's already familiar with programming, they can probably teach themselves Fortran and start being able to make useful changes to your research code in a week or two. Compare, say, C++ where just learning the language enough to be useful could take most of a semester.
Fortran just has better syntax and defaults than C for this stuff. A devoted, expert C tuner with unlimited time on their hands can do anything. A grad student with like a year of experience can write a Fortran code that is almost as good, and finish their thesis. Or, a numerics expert can write a numerical code for their experiments and be reasonably sure that they are operating within a good approximation of the actual capabilities of their machine (if you are an expert on numerical computing and C+assembly, you can write a library like BLIS or gotoBLAS and become famous, but you have to be better than everybody else in two pretty hard fields).
IMO this is important to point out because somebody can bring a microbenchmark toy problem to show C beating Fortran easily. As long as they spend way more effort on the problem than it deserves.
To put it another way, your comment is a little like saying Bash is just as fast as C because you can write shell scripts that inline assembled executables.
This is part of why I'm in favor of things like Rust moving borrow checking into the langauge. In principle you can statically analyze all of that in C++ with a good static analyzer, in practice, it's a constant process of sticking fingers in broken dikes[1] and fighting the ocean. Sometimes you just need a stronger wall, not more fingers.
[1]: https://writingexplained.org/idiom-dictionary/finger-in-the-...
/* They're all the same length */
void add(float *result, float *arr1, float *arr2, size_t len)
{
size_t i;
for (i = 0; i < len; ++i)
result[i] = arr1[i] + arr2[i];
}
float *restrict float_ptr;
https://en.wikipedia.org/wiki/Restrict
It's hard to judge the extent to which this helps. Apparently, when Rust annotated all its mutable references and Box<T> types with the LLVM equivalent of the "restrict" annotation, they exposed a lot of bugs in LLVM's implementation of it, because most C code doesn't use "restrict" pointers as extensively as Rust code uses mutable references and Boxes.
I suspect much has changed since though, so its fields of use may just be convention now.
I also remember from the era, that a one-line program with a '.' in column 6 would generate 600 lines of errors from the IBM optimizing cobol compiler.
My experience working with particle physics code (which is generally C++) is that we could probably make it a lot faster if we completely banned the use of `std::map<std::string, T>`. But since it's there, and since people use it to e.g. look for files or parse configuration, you often find an API that accesses elements in an array by string rather than by index. Sure, it just made your code 20x slower, but it's only one part of much bigger framework and this wasn't on the "hot path" anyway, so you just use it.
Rinse and repeat a thousand times, you can see how things slow down a bit.
https://play.clickhouse.com/play?user=play#U0VMRUNUICdodHRwc...
WRF is one of the most notable...
did a search, ha
- numerical weather prediction,
- finite element analysis,
- computational fluid dynamics,
- geophysics,
- computational physics,
- crystallography and computational chemistry.