brunoalano
In my runs, the shallow 2-layer setting looked mostly similar to AdamW. The more interesting case was moderate depth: around 8 layers, Muon was noticeably more stable and gave better final results. I also saw a fairly large robustness gap under feature noise and edge dropout.
The writeup focuses on the spectral side of the story: singular values, conditioning, and why the effect seems to show up more in deeper message-passing stacks than in the standard shallow benchmark regime.
I included the negative results too: Muon is slower per epoch, it doesn’t win everywhere, and by very large depth the optimizer alone is not enough.
I recently sold my startup (acting as CTO) that worked in the Lead Generation area through Chatbot. Now I'm looking for something new to invest my time in.
My expertise is in the field of artificial intelligence, and with great experience both in front-end (React, Next.js, Typescript) and back-end (Elixir, Python, Rails) since I worked as CTO for many years.
I have a great interest in the following areas:
- Digital Ads, how they can be optimized through AI (mainly when it comes to retargeting)
- Latin America in general
- Note Taking: How AI + Humans can increase productivity and understanding of concepts, something well oriented to RoamResearch (Zettelkasten) but simpler from the user's perspective
But I'm open to other ideas. Contact on HN or email (bruno [at] monoscope [dot] io)
High-Performance in Python with Zero-Copy and the Buffer Protocol
OpenAI Five X Top Dota 2 Players at 3 PST
Arbitrary Code Execution in Core Rust – CVE-2018
Location: Remote / Curitiba, Brazil
Full stack developer, working with Rails and Python, developing SAAS applications, applying good pratices like TDD/BDD. Currently CTO of a Machine Learning and Natural Language Processing startup.
Email: [email protected]
But isn't better use a Functional Language, for example, math functions act like a function in a functional language?
You have some recommendations in math books?
Tips for a young dreamer and developer [Help]
Hello, I've just turned 18 years old and since my 08 years I program (Visual Basic, Python, C, C + +, Ruby, D, Javascript and others) and in these 10 years I have learned a lot!Currently I am dedicated to making startups and projects, and I have great interest in furthering me in math (currently reading "Discrete Mathematics with Applications") and artificial intelligence.My big problem is:-...