txus

At Manifest AI we have just released our open-source CUDA kernels to implement Symmetric Power Transformers, as described in our paper from back in August:

https://manifestai.com/articles/symmetric-power-transformers...

Since this is a variant of a linear attention, you get linear cost when training (as opposed to quadratic in regular attention), and constant when doing inference. This is especially attractive for longer contexts!

Have a look and play with it -- and of course contributions are very welcome! It's an early alpha!

Experimenting is good. I do it all the time. Just take a look at my repos on Github. Just a few of them are conceptually (and arguably) "new", while the others are just exercises to learn about new things. Experimenting about things, even about the basic ones, helps me a lot in my learning process. At a personal level.

But when you watch the bigger picture, with companies actually building software for clients that trust them, then choosing technology is a delicate, non-trivial matter. It's important, and I mean in actual dollars (or euros). There is where the expertise of older developers, people who have founded, sold or shut down profitable and unprofitable companies, who have worked in a gazilion of projects and made a lot of expensive mistakes, there is where that expertise comes to play. They can enlighten the sometimes childish, faction-like debates about technology that keep repeating themselves over and over. That can free us to think about new problems and new things, based on past experience. That's how science advances, and that's how software development advances too, in my opinion.

I chose that word on purpose because of both its semantic and emotional power. Besides, I'm pretty happy that all the critics have been focused on either using the term "cancer" or the swearing at the end, so I guess my points are valid to almost everyone, and you just didn't like the way I expressed them. That sound ok.

The 80% I was talking about.

My article has nothing to do with computer science, but with a human community built around a technology.

This is not (only) a rational argument, but a call to action, and that requires an appeal to emotion.

I'm really proud that so far no one has argued about any point I make in the article, but rather about the "cancer" reference and the swearing at the end. I guess that means I happened to write a pretty damn good article.

I am currently organizing the Barcelona Ruby Conference, and have attended many other conferences in Europe. I know a lot of people put much effort in doing things right. But that's not the majority of the community, unfortunately. I also agree that the Pareto principle (and many other points raised in my post) are a universal problem, not just Ruby's.

I did provide constructive advice. If you read the post thoroughly, you can see a call to improve our learning manners, stay away from hot trends, and even a simple pattern you can use in your day to day to compare technologies, or to understand better some debates (like threads-processes-reactors).