mlsu
- models are trained to improve on some task, e.g. 'improve at math puzzles' is commonly used.
- the best way to improve on some task is to have interpretability into your own neurons (..?)
- if your interpretability is good, you can improve faster - if you have insight into how your own neurons work and can improve them directly
- therefore, models naturally put effort into 'self-interpretability'
I wonder if this is part of the 'secret sauce' of what the labs are doing: giving models the ability to interpret their own weights in the RL loop. In my mind, this is the only explanation for how they could succeed at a task like this. How many blog posts are on the open internet that say "this is each weight of a neural network and what each one means"?
- A TV has a backlight. It doesn't feel as 'natural' to the eye when watching because you are staring directly at an LED.
- A TV is smaller than a projector.
- A TV, when it is off, imposes itself on the room. It's a large plastic/glass object which is obviously electronic. It invades the space.
- Most TV's are full of surveillanceware.
- A TV can be watched even during the day, when it's bright outside. The projector, by contrast, is much less pleasant during the day; I consider this a feature - I should be doing something else when the sun is out. (The only time this kinda sucks is sports. We make do with closing the curtains).
- Movies were made for the silver screen, imho. This might be a hipster opinion but in my view, the medium really does make a difference in this regard; the moviegoing experience is better.
To specifically answer your question - most projectors are just a dumb HDMI cable. Computer, media center PC, apple TV, etc. None (unless I'm not aware of newer models) have an app store, tracking, or other garbage.
This El Nino phenomena is caused by something called Bjerknes feedback. There is a recurring atmospheric 'wave' of relaxed and enhanced trade winds, called the Madden-Julien oscillation (the MJO) that travels slowly west to east across the planet with a period of approximately 60 days. The "active" phase of the MJO usually comes with a burst of westerly (west-to-east) winds in the west pacific. Those slow down the normally easterly (east-to-west) trades a bit. Some years, those easterly trades are slowed down a bit more than usual; this causes the heated up water in the western pacific to 'escape containment': deep warm water moves west to east, in a Kelvin wave. A strong series of these Kelvin waves can cause a mass of deep warm water to build in the eastern pacific. Upwelling there now brings warm water to the surface, instead of cold water, which means the air above the east pacific gets hotter, which means it rises -- low pressure -- which causes the trades to slacken further (-> the air moves relatively in a westerly direction to take up this low pressure), which causes another, stronger Kelvin wave, and so on and so on. This is Bjerknes feedback; it causes El Nino.
These Kelvin waves stop when they hit South America and split: half moves S->N along the N hemi coast, half moves N->S along the S hemi coast. Warm water is less dense than cool water, so there's a big mass of warm water near the coast. This lower density warm water wants to go 'downhill', offshore. Due to Earth's rotation, it's turned right as it travels (since it's in the Northern hemisphere), and then right again, back towards the coast, which balances against the offshore push. The net effect is a consistent mass of warm water held up against the land, and a consistent S->N current along the shore.
When the water is anomalously warm enough, when the El Nino is severe enough, this effect grows to be very large. In San Diego right now the real tide is around 1-1.5ft higher than the "standard" reading from a tide chart. Dive visibility has been incredible, the best on record, since the warm upwelling is drawing from nutrient poor water. Water temps have been incredible. Water temp is 73 degrees in San Diego right now, normally this time of year I'm in a 3/2 fullsuit. Tropical moisture has been off the charts. At a couple points this year, SoCal was the hottest and most humid place in all the USA. The consistency and count of south swells, including waves from tropical storms, has been absolutely insane. Weird currents everywhere, sandbars in crazy spots. Fog, weird bird action. My neighbor's beautiful 50ft tall Monterey pine was singed by this heat, turned brown from the pine beetle, and is probably going to die. I am buying sandbags for the house and big surfboards for this upcoming winter season. It's all just very very strange, I have never seen weather like this ever in my life. Never ever. And I still have decades to go. We'll see this winter, how bad the storms get...
Training and serving LLM's is massively parallel. Your bottleneck is either in energy or chips. China will catch up on chips fully decades before we catch up on energy. Arguably, this is already happening: here in the USA, we have lots of chips, but we are seeing our electricity bills skyrocket as more datacenters come online. We are powering those datacenters with natural gas turbines, not solar, wind and nuclear. Right now, to purchase one of those, you are on a 5-year waitlist. There's your timeline!
China is invested, from a regulatory, cultural, workforce, strategic, and technological standpoint, in an energy abundant future. We are not invested -- and in fact, actively sabotaging ourselves -- on all of those fronts.
Once they acquire chips, we lose, basically permanently. Look at what Huawei is doing recently and what is on their roadmap to understand how fast this catching-up is taking place. I'm sure that David Sacks would say that our innovation motor is pulling their copycat caboose along, as our monopolist tech companies steam ahead towards the frontier. I am not so sure. They are already ahead of us in many technologies. Speaking of technologies, they're building natural gas turbines in house, starting in 2025.
He wants control, so that he can use investor’s cash to do epic stuff like humanoid robots, cyber trucks, an AI that tells you that apartheid was actually cool, rocket cars, real life brain controlled robot bacon chungus, etc.
“Other shareholder meetings are snoozefests but ours are bangers. Look at this. This is sick.”
Elon’s pay package: he gets 400 million Tesla shares if he raises the market cap of tesla(xspacextwitterxAIx) to $8.5 trillion (nominal). He spent a cool 250 mil on making friends with the republicans, so as to ensure that his incoming epic awesome merger is not blocked.
A merger, plus the amount of inflation this government has already produced over the next decade, will probably see X co hitting Elon’s salary goals.
Datacenters in space are visionary -- they are maximally energy rate dense and they expand the area of energy absorption. Might be 100 years too early but that is where we are headed, after we turn all of the available biosphere of earth into computronium, that is.
The only superior technology left would be von Neumann probes.
Oh well, at least the site itself is technically excellent.
If you listen to them, and you don't have to listen very hard to hear it, basically everyone at these labs is telling us that this technology is extremely dangerous and should be slowed down or paused entirely. Yet, they, the only entities with the power to actually do anything about it, are not acting AT ALL as if that's the case. They are all barrelling forward as quickly as possible. RSI, THE number one risk according to these guys, is being adopted at breakneck pace up and down the stack, from designing silicon, to training, to inference.
It's ridiculous and insane and I believe can be accurately summed up as, they are being assholes, because if they are actually right about this we are all gonna die. At the very least, and far more likely, every fun creative expressive human thing that is machine legible will be replaced by a torrent of machine slop. It's not "benefiting humanity." These mathematicians are telling you it's not benefiting humanity. It sucks.
Alignment problem.
You would not have time because you'd be fired before you finished. That's what's stopping people from doing projects like this today.
Despite nobody at openAI thinking of themselves as an asshole; despite society urging openAI not to be an asshole; despite the fact that being an asshole is entirely unnecessary even to accomplish whatever objective they are setting out to accomplish; despite everyone at openAI loudly declaring: we are not assholes!
They are still assholes.
When you don't have any insulin, your body goes into a state of diabetic keto-acidosis (DKA). While in this state, sugar builds up in your blood, but does not enter your cells. Your liver dumps glucose into the bloodstream which causes dehydration, as the kidneys work overtime to clean the blood. The body eventually uses an alternate metabolic pathway to produce energy which dumps byproducts into the blood which cause it to turn acidic among other things.
Speaking from experience, severe DKA is extremely uncomfortable. The problem is in your blood, so you feel a constant and extreme discomfort everywhere in the body, there's no body part that you can focus on where it isn't happening. You become extremely tired and lethargic and also irritable. You get angry and panicked and cognition slows down so you don't feel like yourself. You are very hungry and very thirsty but nothing you eat or drink alleviates the sensation. You also feel as though you cannot breathe enough to sate your body, so your breathing becomes faster and shallower.
It's difficult for me to imagine anything worse than dying of DKA. Maybe being lit on fire? That's short though, a death by DKA happens over hours or days. The cruelty here is spectacular.
Insulin is on the WHO essential medication list. The type of insulin that could alleviate these symptoms is cheap and widely available. There is absolutely no excuse here. Every doctor and nurse knows how to treat this and stop this kind of suffering.
We were flabbergasted. Asked the tech what was optional or what was required as part of our visit and he said, oh, just the top line item for the exam. $95.
It has to be illegal to do that. I feel like next time I go in there they are going to give me the ol' Clark Stanley runaround.
What's interesting to me is that this can be completely true (I believe it is), but why can seemingly nobody see the value and prioritize saving it. Opera is beautiful, it's one of the many things that make life worth living.
More efficiently allocating capital ought to be just a way to generate more resources for producing opera. And yet...
Metabolism: The chemical processes inside a body that turn food or nutrients into energy. LLM's do not spontaneously do this, they are powered by plugging them into the wall.
Growth: The ability to get larger and develop over time. LLMs are fixed in size (and in fact don't really have a size, because it's a computer program) and do not grow or change over time.
Reproduction: The ability to create new organisms. LLM's do not reproduce themselves.
Response to Stimuli: The ability to react to changes in the environment. LLM's do not have an environment. Their environment is a man-made, theoretical structure of logical operations implemented in silicon.
Evolution: The capacity of a genetic system to change and adapt across generations. LLMs do not change or evolve over time.
So... 0/5! Big fat goose egg for LLM's.
Like, it ought to be enough to say that the problem with AI is that it’s not human. Unfortunately, we live in a world where many (most?) people have willingly put away the childish things, so as to achieve some goal.
The problem is that the point of life is to explore and have fun and have human to human experiences. I think math is probably fun because you’re working on hard problems with your collaborators.
It should be obvious to everyone, as it is to children, that the point of life isn’t the goal, the point is the experience one has of living. They say that when you die the only thing you lose is the current moment. The problem with AI is that it doesn’t have a current moment to live in or to lose, it’s an inanimate thing. So, it should be obvious to everyone that the problem with the AI is that it’s not human. Alas.
It’s just silly talk. My dad works for Nintendo and he can give me a gold foil pikachu any time he wants, also, he can delete your Pokémon any time he wants because he has admin access on the Nintendo server. And also he can beat up your dad because he has a black belt. Oh your dad has a black belt? Well actually I lied, he has a rainbow belt which means he can beat up your dad still.
Like this is silly right? It’s insulting. We both know that there are live nuclear weapons whose coordinates point at my city and at your city and the only thing that stops them is two keys and a button. Nuclear weapons, you know those things that have killed hundreds of thousands of living men women and children? There is no room for speculation in that regard: my city has a navy base and there is a live nuclear weapon pointed at it, myself and my entire family will die if it goes off. Now contrast: ahem.
Oh sorry my dad also has like a super secret rainbow STEALTH powers that could actually totally insulate my house from nukes. If you think about it, I actually have no reason to be concerned.
If that is true then this seems to be, again, a case of AI producing an interpolation over data it has seen before. Everything about openAI's behavior indicates that they were using the transcripts as input. Why not have the AGI choose a different Millenium prize problem?