No GCC support if I read that correctly and the effort is expected to take months of full time work. I wonder what that'd break or is Clang significantly more common on OSX now?

Well, for one, this means that Apple Silicon has no Fortran compiler, which breaks a decent chunk of the scientific computing ecosystem.

Won’t this work? https://lfortran.org/

I just did a quick google search for it. I know nothing more than that it is FORTRAN compiler for LLVM. (It claims to be an alpha release)

Perhaps! It's one of the two candidates that might actually work in the near term. It didn't quite seem to work out of the box and given the newness of the project, we are a bit cautious, but there's some folks looking into it, to see if they can work through the issues.

Not sure if it's changed, but last i checked a few years back, most of the BLAS/LAPACK etc. stuff was f77, which should 'do' with f2c (albeit, perhaps not as fast) and is at least partly why f2c was hosted at netlib..

I had no idea, that explains so much! I was wondering how these open source numerical systems were building fortran code "so well", as in not a huge operational burden. My understanding is that the OSS fortran compilers are used for dev/debug, but when it comes to production, commercial Fortran compilers for your platform vendor offer much better performance.

Would it would work under Rosetta 2 (Apple's x64 emulation/translation layer) until the aarch64/darwin ABI can be ported to GCC? Or is there a reason to assume it might not?

Yes, it works mostly fine under Rosetta. However, it is also precisely the area where you don't really want a translation layer, because you're spending a lot of time doing heavy compute.

That makes sense. I'm hoping it's going to be an early adopter problem that'll have been solved by the time the transition is deemed complete.

Many compute heavy tasks are using Metal and those should continue to function at the full speed of the hardware.

I doubt GCC is, or even could be, using Metal.

I am referring to modern HPC like workloads, they will be using the GPU. Not saying GCC is using Metal in any way whatsoever.

HPC on macOS? Using Metal instead of CUDA/OpenCL? Sounds extremely rare.

is there not an flang project? Is it any good?

There's several! We tried all of them without much luck :(. The most complete seems to be the old flang, which was basically an open sourced version of the PGI compiler hacked up to generate LLVM IR by concating strings to generate textual LLVM IR. However, that project is basically abandoned and also depends on LLVM 7, while you probably want at least LLVM 10 to generate good code for Apple Silicon, so somebody would have to do that porting work, which doesn't seem like a worthwhile investment.

Hopefully apple will do something about it as they did advertise their new system would be great for machine learning and all, which seem to be moot without many popular numeric libraries that directly or indirectly use fortran. Would be great if clang/llvm suddenly gain a great open source fortran compiler just because apple moves to arm.

This is incorrect, the LLVM project has an official Fortran compiler in tree called flang: https://github.com/llvm/llvm-project/tree/master/flang

It is quite hard to write portable Fortran code, but code that at least compiles with PGI should compile with that as well.

If you actually read the code, it turns out that said compiler is not yet capable of generating LLVM IR let alone binaries.

Wait what? We were using the old flang project just fine. I thought they just moved it in tree and keept developing it.

There was old flang which was basically just the PGI compiler stripped of its backend and with a piece bolted on that converted the compiler's internal IR to LLVM by basically dumping out strings and then parsing them back in. Many people in the LLVM community thought that this was not a maintainable long term strategy for a Fortran frontend, and after much debate, the flang team decided to start over on Greenfield compiler called f18, which was to be architected more like clang. Then Chris released MLIR and said team decided to target that rather than LLVM directly. The only thing that was kept from old flang were some of the runtime libraries. f18 was recently merged into the llvm-project repository whereupon it was renamed flang. Old flang is dead, but the terminology is a bit confusing now. Additionally there are two external LLVM Fortran compiler efforts: lfortran out of the US national labs (which also paid PGI for flang) and fc, which I don't believe is currently open source, but is being developed by as compiler consulting shop on India called CompilerTree (not sure who's footing the bill for that one). Anyway, long story short, none of them seem to work. Old flang generates binaries, but is abandonware at this point, so not the correct foundation to build on. New flang doesn't even generate LLVM IR yet. FC doesn't seem public yet as I said. Lfortran may work, but the authors say it's very alpha and it certainly didn't work out of the box. I know some folks are looking into it to see what would be required.

I wonder why this is allowed to persist? Have there been any studies across benchmark suites relative to compiler and platform? What was the spread? Was this general purpose, it works well enough speed or was this I need to get 1.75% better on task x or my super computer budget is blown?

I find the whole Fortran ecosystem and lineage fascinating for some reason. Like an alternative universe where integers are only used for loop counters.

Apple sucks for scientific computing. It does not let you even load your own generated shared files, because Apple determines it is too dangerous for the health my own purchased computer. I have never had that problem on Windows or Linux. That said, you can get gfortran by as simple as typing brew install gcc-9, or some similar command (requires brew installation).

Does anyone in the scientific computing ecosystem use Macs?

I can't think of...any program in that sphere that aren't Windows or Linux?

Sadly, yes. I've clients who still prefer to work on their Mac laptop and desktops. Large runs still go to Linux clusters, but small, debugging runs occur locally.

Per the parent, the lack of Fortran compiler causes an issue because some BLAS/LAPACK libraries align their symbols with whatever a Fortran compiler would do. As such, unless there's been a recent change, things like CMake need the compiler to determine the name for linear algebra routines.

I phrased the above as "sadly, yes" because supporting macos has been a pain and costs a fair amount of labor. In my case, Homebrew is required and there a bunch of specialized quirks in the build and packaging system that I'll contend are more difficult to deal with on macos than on Linux and Windows. Lastly, the requirement that macos be virtualized only on mac hardware complicates automating the build and test system since those processes must be relegated to a macos machine and I'd really rather not do primary development on a mac. Frankly, I'd pay Apple a not small amount of money to virtualize macos legally on non-mac hardware.

BLAS/LAPACK isn't that much of an issue, since Apple probably will need to ship their own implementation of this for Apple Silicon to get reasonable performance, and they can implement this in C, assembly, or whatever they want.

If they don't do that, then you can just use any "generic" C BLAS/LAPACK implementation, performance will be poor, but that's pretty much the only option if your hardware vendor doesn't support BLAS.

I guess people can write their own BLAS as well, but if you want to re-implement BLAS in 2020, then Fortran is probably not the best language to do that.

Apple's copy of LAPACK has not been keeping up with upstream and is missing some of the newer methods and bug fixes. Also while those are certainly the big two dependencies they are by far not the only ones.

I see the opposite from you: Mac laptops, crunching under Linux, very rarely see windows in the wild. I know windows sales are still robust so this is what I see but am not sure how far it can be extrapolated.

A huge chunk of scientific and numerical libraries rely on Fortran, for which 64-bit support is seriously lacking on Windows.

Doing this work is significantly easier out-of-the-box on Mac/Linux device.

Data Science is my main job and the first thing I did at my last job was ask if I could wipe windows and put Linux on the desktop they gave me.

> A huge chunk of scientific and numerical libraries rely on Fortran, for which 64-bit support is seriously lacking on Windows.

How so? mingw-64 exists, and so does Intel Fortran (with good integration with the VS IDE as well).

Intel and PGI Fortran work just fine on Windows, with VS integration as pointed by others already.

Lots of students have Mac notebooks, but want to run julia/scientific python/etc.

Yes. Most people in my lab have a MacBook (Pro) where we do most of our coding, and we ssh into a Linux cluster to run most of the compute. We have desktops but these are all Windows/Linux dual boots. We all practically need to use PowerPoint as well, so Mac is nice to have here since it can’t go on Linux laptops, and nobody wants to constantly reboot all the time.

Walking down the hallway in my physics department (pre-covid lockdown), I see almost all Macs. Some of them run windows, but most are apple hardware and software. Windows is quite rare when it comes to professors. It's more common among the grad students though, about tied with MacOS.

It was like this in my previous department as well.

Yes, that would be me, and half if not most of my colleagues.

Very much yes

Sure, lots.

By Homebrew policy, anything requiring a C/C++ compiler but doesn't compile with clang is out. So gcc not working shouldn't disrupt the overall Homebrew ecosystem too much. The biggest problem is gfortran, which is what the great majority of gcc dependents use it for (just `rg 'depends_on "gcc"'` in homebrew-core and all the "# for gfortran" comments should make this obvious).

Btw, here's the list turned up by a dependent search in core:

  $ brew uses --recursive gcc
  abyss                           dynare                          itk                             numpy                           qrupdate
  adios2                          eccodes                         itpp                            [email protected]                      r
  apache-arrow                    ensmallen                       jags                            nwchem                          ratfor
  apache-arrow-glib               fastme                          joplin                          octave                          rawtoaces
  arcade-learning-environment     fftw                            json-fortran                    ola                             raxml-ng
  armadillo                       field3d                         kahip                           ompl                            reprepro
  arpack                          flann                           kallisto                        open-mpi                        root
  arss                            fobis                           kim-api                         openblas                        scalapack
  astrometry-net                  gdal                            lammps                          opencoarrays                    scipy
  asymptote                       gdcm                            lapack                          opencv                          shogun
  aubio                           gmic                            libbi                           opencv@2                        simple-tiles
  augustus                        gmsh                            libmatio                        opencv@3                        siril
  biosig                          gmt                             libxc                           openfast                        skymaker
  boost-mpi                       gmt@5                           liquid-dsp                      openkim-models                  sratoolkit
  bsponmpi                        gnuradio                        luaradio                        osi                             suite-sparse
  caffe                           gr-osmosdr                      mapnik                          osm2pgrouting                   sundials
  calceph                         grace                           mapserver                       otf2                            superlu
  cdo                             graph-tool                      minimodem                       packmol                         synfig
  ceres-solver                    gromacs                         mlpack                          pcl                             urh
  cgns                            gwyddion                        mpdviz                          pdal                            veclibfort
  clp                             hackrf                          mpi4py                          petsc                           vips
  coinutils                       hdf5                            mpich                           petsc-complex                   virtualpg
  cp2k                            hdf5-mpi                        mvtools                         pgplot                          visp
  cpl                             [email protected]                       ncmpcpp                         pgrouting                       vtk
  csound                          [email protected]                        nco                             plplot                          waon
  dartsim                         hypre                           ncview                          pnetcdf                         wxpython
  datetime-fortran                imake                           nest                            points2grid                     zita-convolver
  dlib                            imgproxy                        netcdf                          postgis
  dungeon                         inspectrum                      networkit                       proteinortho
  dvc                             ipopt                           ngspice                         qd

147 formulae.

Edit: Forgot to --include-build. That would add 4:

  boost-python3 btfs libtorrent-rasterbar openimageio

libtorrent-rasterbar made front page today btw...

Weird to see wxPython on the list..

Any numpy (which uses gfortran) dependent is on that list. I suspect some could partially work by sacrificing some functionality, or maybe just run slower (disclaimer: I have no idea of numpy internals).

Interestingly at WWDC Apple said that they are contributing patches to NumPy to get it to work on Apple Silicon

Wouldn't that entail swapping out all the underlying Fortran that makes NumPy so lovely?

does numpy leverage blas?

Yes. https://github.com/numpy/numpy/blob/62fa23c44fb49c1d238e1de4...

well, that's probably where the fortran dependency could creep in, no?

yes

GCC supports ARM targets and the macos platform. It took a couple of hours to add support for the new target triple to Rust, so it should take about the same work to add support to it for each of the GCC frontends (10x couple of hours ~ couple of days, maybe a week).

You don't need to implement neither hardware support nor OS support, just combine the two things that are already implemented, and fix a couple of incompatibilities, but not many (some parts of OS support will be x86 only and these would need fixing, but fixing these should be trivial since you can c&p from the arm+iOS code).

Why would it take months to add support for this to GCC? GCC's architecture would need to be really flawed for this to be true, i.e. , requiring you to re-implement a big chunk of hardware support for each OS, or full OS support for each new hardware. Given how many architectures and OSes GCC supports I just can't imagine this being true.

Unless by this work taking months, you mean "not doing anything the first 99% of the days, and doing all the work in the last couple of days". In that case, sure, this can take as long as you want.

Yeah, I was surprised by that statement too, but then again, I know nothing about how GCC is architected. I'm hoping somebody will come along and just fix it in a few days, but who knows.

IMO Apple should be sponsoring that work.

During the ARM announcement they said they would be contributing support to open source projects, and Homebrew was on the slide, along with a dozen or so other major projects (though Homebrew itself represents hundreds). I do not know what form that assistance will come in.

Apple has been dropping lots of balls in this way.

They should support open source in an official way. They should support other languages besides swift. They should support virtualization/containers. They should embrace techies, scientists and engineers. But it seems they're closing down these areas of their ecosystem.

IMO Mac OS X should come with its own integrated package manager and corresponding central repository for distribution of various open source software packages. A package manager that worked well with the App Store and Xcode.

This is their only real chance to keep Devs from migrating to Windows 10 with WSL2 or baremetal Linux. I couldn't believe they didn't announce any thing for Big Sur, I could have sworn they would if they really cared about non-macOS/ipadOS/iOS devs. Especially that they're also dropping Perl/Python/Ruby frameworks.

If anything, they could have just low-key embrace brew and take it from there. They didn't, and I think that sends a message: they've given up.

Starting to feel vindicated by my refusal to use Homebrew when I migrated to macOS recently. It felt like bringing a whole lot of baggage along.

All I need is working golang and Python 3. The rest I can do on a box in AWS.

It's nice to have a setup that lets you get stuff done with a remote machine without needing a lot of parts strung together - I had this at a previous role. I've also had roles with very little support, and being able to spin up all the utilities I needed on a work computer was helpful.