suziemanul

We're hosting a boxing-style debate in San Francisco on May 5th between Łukasz Kaiser, co-author of "Attention Is All You Need", and researchers building post-transformer models.

I'm the CEO of Pathway and co-author of the Dragon Hatchling (BDH), our post-transformer architecture. We're organizing this to bring key researchers on both sides to debate each other without any BS. We invited Łukasz specifically because he can push back on the post-transformer thesis harder than almost anyone.

Confirmed so far:

- Łukasz Kaiser, co-created Transformer, ChatGPT, o1, o3, TensorFlow - Mathias Lechner, Co-founder/CTO of Liquid AI, co-created liquid neural networks (MIT CSAIL) - Adrian Kosowski, CSO of Pathway, created BDH (ex Inria, École Polytechnique)

Moderated by me, Zuzanna Stamirowska, and Dex Horthy.

Tuesday, May 5, 5 to 8 PM PDT. San Francisco.

Goal: Debate upfront without panel pleasantries. Answer core questions like: what the Transformer got right, if and how it's hitting limits on long-horizon reasoning and memory, and whether the alternatives like liquid networks, world models, and BDH can solve these limitations to define the next frontier.

Additional context: WSJ predicted AI will outgrow LLMs in 2026, while mentioning the post-transformer research by Pathway and the world models from Yann LeCun and Fei-Fei Li as defining shifts for the year. This debate builds on it along with BDH's recent result on hard constrained problems (97.4% accuracy on Sudoku for a language model)

This is still the missing piece of the puzzle.

Yes.. we have 60% that inception will happen within 24 months.

In this video Lukasz Kaiser, one of the main co-authors of o1, talks about how to get to reasoning. I hope this may be useful context for some.

https://youtu.be/_7VirEqCZ4g?si=vrV9FrLgIhvNcVUr

Some folks say it's not Fortune 100 but Fortune 1 ;-)

Hi I am Zuzanna, CEO of Pathway.

A use case we have been working with data scientists is to create dashboards sourced from live data, e.g. stock ticker, logs, chats, news.

Why do I think it’s cool?

- you can run algorithms and analysis on streaming data (from Kafka, RedPanda, Azure/EventHub etc.) directly in Jupyter, - you get an interactive way of working and prototyping with live data, - you can do replays on static data.

Yeah... Curious to see how this will unfold.

Totally, most of the Transformers folks ;-)

I remember seeing short stories generated back in 2020 and they were sort of cool but not that great.

Scaling of training was the challenge back then (of course).

Google was already too corporate. Please remember that Sergey Brin and Larry Page were no longer at the steering wheel back then. I have been told that it was also a cultural issue linked to "delivering brilliance". Simplifying: Google promoted tiny teams or individual contributors building things that had to become a massive success quickly. Open AI took a number of hand picked brilliant people and let them work together on a common goal, silently, for quite some time.

Some companies just have an unfair advantage. A certain magic. And OpenAI's magic is at risk right now.

This is a handful of people we are talking about. The top algorithimic world is incredibly small.

In short, either they didn't or where unable to create a favorable enough environment for this to flourish.

Look where for example Lukasz Kaiser is now [OpenAI]. Google had a culture issue when it came to "delivering brilliance". It was a bit "you do it as a singleton contributor" or you don't. OpenAI put a number of such brilliant people working together on one goal, silently, for quite some time, and we all see the results.

This is Olek Madry and Jakub Pachocki we are talking about. Check out their respective dblps if you don't get it. It's a kind of loss that will be hard to recover from.

In relation to other comments here. There is "coding" and there is "God's spark genius of algorithms" kind of work. This is what made the magic of OpenAI. Believe me, those guys were not "just coding". My bet is that it could be all about some research directions that were "shielded" by Sam.

As for the stable coins, the researchers from the University of Chicago claim that their stability is a bit more nuanced. Especially, even a stable coin cannot defend against a run https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4226027