yaroslavvb

They spent a lot of effort on consistent naming. It's basically equivalent of hundred of open source math/visualization packages merged into one, and a single person has made sure for 30 years that functionality integrates in a consistent way. It's much less overhead to remember that using a large collection of OSS libraries. So the naming structure in itself is valuable

I like this project! I've used Wolfram Language since high-school and collected a large number of notebooks that I revisit sometimes. I'm curious, what fraction of cells work under Moxi? 1,275 notebooks, 3GB archive here -- https://drive.google.com/file/d/1RRqHnnCUevm7E0zKsHQvKNKxpzZ...

You can use eigenvalue solver to find singular values of any matrix but not vica versa. Implementing eigenvalues needs complex numbers

Eigenvalue are more flexible than singular values -- If you have procedure for computing eigenvalues of C, squared singular values of X are eigenvalues of C=XX'. Left and write singular vectors of X and eigenvectors of XX' and X'X. For rotation rotation matrix, singular values are all 1. Meanwhile eigenvalues tell you angles of rotation https://math.stackexchange.com/questions/4874616/obtaining-a...

I had similar experience, Claude made report of MRI for achilles tear, it measured the gap, but it was completely hallucinated. Achilles tendon is black on the MRI, it instead measured 13mm distance between two completely different things (looked white), the radiologist looked at and saw no gap at all

its real, I tried it a couple of weeks ago, asked some questions and got answers - https://x.com/yaroslavvb/status/2062692318415867983

This project gave me motivation to build the deep learning next token prediction integration for JetBrains because I was using PyCharm at the time. (Eventually, it wasn't continued because it was kind of expensive to host)

I don't hold grudge, GPT-2 wasn't that great of a model, so releasing it would be more of a publicity value. But the blog post already had that purpose.

Thanks for the share! Didn't realize eleutherai launched around same time

This was around the time I trained Transformer-XL (outside of OpenAI) with Ben Mann (https://yaroslavvb.medium.com/scaling-transformer-xl-to-128-...) . Originally we wanted to release train and release the weights as a kind of GPT-2.5, but our OpenAI friends pushed us to keep weights closed.

Modern AGI discourse defies the voice of reason

I used to look at all TensorFlow questions when I was on the TensorFlow team (https://stackoverflow.com/tags/tensorflow/info). Unclear where people go to interact with their users now....Reddit? But the tone on Reddit is kind of negative/complainy

Stricter (but not looser) standards can be imposed on state level. Canada has no binding national drinking water law, they leave it to territories/provinces to decide how to implement guidelines.

Balancing protection against water bills - https://www.epa.gov/newsreleases/epa-announces-it-will-keep-...

I've used this feature before to make my chats discoverable through search engines. I had to manually click it each time I shared, it didn't toggle on.

Researchers like to talk about and show off their work outside the company. If you don't let them, they get unhappy and leave.

The difference is that for some assets you can calculate value based on fundamentals. IE, humans need shelter, hence estimate future value of shelter (real estate) based on migration trends and other factors. How do you estimate future value of bitcoin? Lack of predictability is probably why serious investment funds don't go into crypto

Realistic simulation of neurons is expensive. Back in my grad school days we ran Genesis and could afford at most 10k neurons - each neuron needs a lot of work to model the corresponding differential equations. However, it's unclear how to translate this into requirements for artificial neural networks -- the type of computation is too different.

A different metric is a more relevant goalpost -- number of synapses. If each of 125 trillion synapses in the brain can adjust its strength independently of others, it loosely corresponds to a parameter in a neural network. So if we get 100 trillion parameter networks training but still no human intelligence, we'll know conclusively that the bottleneck is something else. Currently training 1T parameter networks seem feasible

It has a name too -- крякозяблики (kryakoziabliki)

Haha, me too. I guess it's a good learning exercise. I've been using Mathematica for 20 years (my name still shows up on https://stackoverflow.com/tags/wolfram-mathematica/topusers), but in a time crunch I would do this in Python.

SF has programs that supplies methadone to homeless, ie, www.sfstreetmed.org. This doesn't fix underlying behavioral disorders.

1. You don't have to sell the stock -- one could fund an ultra-wealthy lifestyle by renting expensive things, using stock as collateral. As your stock goes up in value, so does the value of things you have access to.

SPSA seems to be missing. OpenAI's "evolutionary strategies" paper used a form of SPSA. https://en.wikipedia.org/wiki/Simultaneous_perturbation_stoc...

On 64 core Xeon E5 this example gives 1.8x increase in user time, but 8x decrease in wall-clock time

I was an engineer in Google Books in 2011 and got an impression that this effort turned out to be not as useful as expected. This took away energy from the legal fight. Having access to every printed book seemed amazing in principle, but it didn't translate to amazing demand for the service. My theory is that it didn't lower the barriers to knowledge much -- there's not that much useful information left in offline-only storage.

Can you elaborate on the performance-per-dollar part? IE, I'm seeing GCP providing V100 at $2.48/hour and a TPUv3 at $8.00/hour