kk58

AI for science is much bigger than RL or Generative AI in science.

There are several classes of models Like operator learning, physics informed neural networks, Fourier operators

That perform magnificently well and have killer applications in various industrial settings

Do read the attached paper if you're curious about AI in science

Neural ODE reframes this: instead of focusing on the weights, focus on how they change. It sees training as finding a path from untrained to trained state. At each step, it uses ODE solvers to compute the next state, continuing for N steps till it reaches values matching training data. This gives you the solution for the trained network.

https://ourworldindata.org/emissions-by-sector

The major emissions sectors are agriculture, transportation, cement and metal, energy usage in primary sector for thermal processes.

For someone who seems to have dedicated their lives to climate and AI solving energy intensity of deep learning isn't even relevant let alone a factor in emissions, also better climate models aren't the solution for reducing emissions.

We need not only new tech but a new socio-technical system for reduction of emissions. Rice cultivation, cement, waste, heating of houses in northern latitudes all generate sizeable quantum of emissions what's AI going to do there to solve these?

Optimization can squeeze some gains but we need revolution in fundamental science and engineering capabilities to quickly discover, synthesize and scale the manufacturing of new materials whilst rewiring the every elememt of the the supply chain to support such transition. This is a problem for AI- yes, but is this where bulk of climate AI research is focused on? Sadly not!

What is more motivating and upbeat though is the fact scientists from traditional engineering domains are increasingly discovering AI. SciML work by Chris rackauckas PiNN by karniadikis, neural operators by anima Nvidia are some of the best forays into this space. I have not mentioned the frentic research happening in AI and microbiology which can be a game changer in invention of biotechnology solutions for climate change.

Yet sadly none of this is ever mentioned or covered in climate AI related conversations.

Now what's trippy is this node has position data. So node feature and position it appears is used to create a operator that projects a sequence to a semantic space.

This seems to work for any modality of data.. so there is some thing about order in which data appears that seems to be linked to semantics and for me hints about some deep causal structure being latent in LLM

In transformer what do query key matrices learn? How are their weights somehow working to extract context no matter which word appears in which position?

What is clear is that on this earth itself we have cetacean, corvid, cephalopod intelligence which is wired very differently. Perhaps we need to respect the diversity of intelligences that exist and study this growth in LLM and adjoint areas as just synthetic intelligence.

Rebranding maybe could help drive a level of objectivity this conversation on ethics etc that seems to be missing

I'm working on this.DM me if you're interested

This is a book on how to learn anything and nothing really about guitar. It distills the zen approach to life in a very usable way. I made a major career shift to ML research using the lessons I learnt from this book

The Gita It helped me develop depersonalised thinking and to some extent the ability to view people equanimously. I feel it has helped me accept and become a lot more inclusive person than who i was before i read it