In the NLP space there's been a lot of work recently around reducing model sizes, since they've started to reach the point where model weights sometimes don't fit in the memory of most GPUs.
There's also projects like MarianNMT which completely abandon Python and write heavily optimized models with fast languages that can run quickly and accurately even without GPUs. I think we'll see a lot more of this, though of course there's a pretty big barrier in the sheer rarity of being good at both deep learning research and writing optimized low-level code.
As for writing low level code, I thought that was something usually handled by the compiler or where even the advanced high performance for high price mostly tweaked the compiler after analyzing the output. Not my direct space so I speak with no authority.
Constraints are the mother of creativity.
I don’t see something new here, these institutions to encourage people to share are old, so it must be a problem that had been recognized for a while.
Aisde from some of the academics and the "gain and share knowledge for knowledge's sake" types they hire why would they care?
For the record, I don't like the idea of scientific research becoming proprietary. At all. But is there anyone credulous enough to think these organizations would willingly risk their bottom line for principles like "openess" and not just play the PR games to make themselves appear open and concerned?
In other words "Don't LOOK evil but do evil when no one's looking".
The Frances Haugen already shows how damaging such openness can be.
If academics want to do research on expensive cutting-edge tech, they will have to join industrial labs or pool together resources, similar to particle physics or drug discovery research today.
I suppose the same will soon be true for most ML-related areas of research sooner or later, at least as far as applied ML is concerned.
Already, a substantial amount of research innovation in NLP and CV has been coming from big companies in recent years.
Of course there is a discussion to be had about what that means for society at large. At this point, a lot of said companies to publish their results at conferences etc. But what if at some point they decide to be as "open" as OpenAI (ie., not)?