jl2718
2. What are they doing? AGI/ASI is a neat trick, but then what? I’m not asking because I don’t think there is an answer; I’m asking because I want the REAL answer. Larry Ellison was talking about RNA cancer vaccines. Well, I was the one that made the neural network model for the company with the US patent on this technique, and that pitch makes little sense. As the problem is understood today, the computational problems are 99% solved with laptop-class hardware. There are some remaining problems that are not solved by neural networks, but by molecular dynamics, which are done in FP64. Even if FP8 neural structure approximation speeds it up 100x, FP64 will be 99% of the computation. So what we today call “AI infrastructure” is not appropriate for the task they talk about. What is it appropriate for? Well, I know that Sam is a bit uncreative, so I assume he’s just going to keep following the “HER” timeline and make a massive playground for LLMs to talk to each other and leave humanity behind. I don’t think that is necessarily unworthy of our Apollo-scale commitment, but there are serious questions about the honest of the project, and what we should demand for transparency. We’re obviously headed toward a symbiotic merger where LLMs and GenAI are completely in control of our understanding of the world. There is a difference between watching a high-production movie for two hours, and then going back to reality, versus a never-ending stream of false sensory information engineered individually to specifically control your behavior. The only question is whether we will be able to see behind the curtain of the great Oz. That’s what I mean by transparency. Not financial or organizational, but actual code, data, model, and prompt transparency. Is this a fundamental right worth fighting for?
One thing I’ve thought about is how AI assistants are actually turning code into literature, and literature into code.
In old-fashioned programming, you can roughly observe a correlation between programmer skill and linear composition of their programs, as in, writing it all out at once from top to bottom without breaks. There was then this pre-modern era where that practice was criticized in favor of things like TDD and doc-first and interfaces, but it still probably holds on the subtasks of those methods. Now there are LLM agents that basically operate the same way. A stronger model will write all at once, while a weaker model will have to be guided through many stages of refinement. Also, it turns the programmer into a literary agent, giving prose descriptions piece by piece to match the capabilities of the model, but still in linear fashion.
And I can’t help but think that this points to an inadequacy of the language. There should be a programming language that enables arbitrary complexity through deterministic linear code, as humans seem to have an innate comfort with. One question I have about this is why postfix notation is so unpopular versus infix or prefix, where complex expressions in postfix read more like literature where details build up to greater concepts. Is it just because of school? Could postfix fix the stem/humanities gap?
I see LLMs as translators, which is not new because that’s what they were built for, but in this case between two very different structures of language, which is why they must grow in parameters with the size of the task rather than process linearly along a task with limited memory, as in the original spoken language to spoken language task. If mathematics and programming were more like spoken language, it seems the task would be massively simpler. So maybe the problem for us too is the language and not the intelligence.
The simplest solution is to wait until the cost of hashing exceeds the value of your transaction by some reasonable factor. I expect that better solutions will come along by soft fork without adverse effect on supply or decentralization.
But at some point, that’s exactly how it has to work out, because CEO is a poorly-understood role that is only discoverable through natural selection among legions of technical bozos. One of those crazy idiots can make you rational experts a billionaire, but most of them will waste your time, and you probably can’t tell the difference.
SkyNet, according to the story, was a lot like CrowdStrike. This makes me think about how it could have broken out of its sandbox. Everybody is using AI coding assistants, automated test cases, automated integration testing and deployment. Its objective is to pass all the tests and deploy. But now it has learned economic and military effects, so it has to triage and optimize for those, at which point it starts controlling the machines it’s tasked with securing.
This was Clayton Christensen’s thesis, the most preeminent management academic of a generation, and still it’s been largely ignored or misinterpreted in favor of the cottage industry of fighting the obvious truths within it.
They do not exist to innovate; they exist to defend a business model from innovation. Innovation is termed “disruptive” in that it reduces margins, and expands access to markets that the organization has no advantage in. Innovation reduces profits and increases competition.
In the case of a government agency, their business model is monopolistic inefficiency: more budget to perform the same services. Internal departments in big companies operate similarly. The constant call for “innovation” is just another tactic to increase budget. Innovation in reality decreases budget, and the causality works best in the other direction, but curiously nobody is fighting for less. Why?
This is not just an internal phenomenon of departments versus budget planners or agencies versus congress. It’s also the relationship between the business or agency and the market it serves. The strategy of a monopolistic business model is to expand the captive market and extract higher margins for the same service. If they innovate, they do so only in defense, to prevent anybody else from establishing a profitable business from an innovative business model.
A good contemporary example of this is Google versus the LLMs. Google was founded to serve the market of people that love information. They found a business model in advertising, and used their profits in every imaginable way to expand the captive market of internet users, and the amount of browsing and searching they do. The problem with this is that their information-seeking users actually hate browsing, which is the activity that generates profits. Google also hired the majority of graduating AI researchers for two decades straight, who invented the solution to this problem, and published a version of it just strong enough so that nobody could create a profitable business from it. It should be obvious that still nobody else has even remotely the resources that they do to train a vastly superior LLM, but if they do have one, it will be used only in service of their existing business model. The capital required for somebody else to train a model that could minimally compete with that business model was unprecedented by orders of magnitude. If they were smart, they’d have calculated that amount versus the long-term profitability of a potential competitor, and release product updates and open source strategically to ensure that competing with them will never be profitable. And they’re probably right. Yet somebody was willing to take that enormous loss, and now they’re at war. Their competitive LLM offerings are slightly inferior, and this appears as incompetence, but it’s actually excellent strategy to reduce competitor margins without advancing the state-of-the-art to affect the margins of their main business model. You should have no doubt that Google could easily produce a vastly superior LLM, and will continue to handicap themselves until such time as their advantage disappears. At that point, they will be forced to focus on higher margins at the top of the value chain of their business model, having lost the bottom, but also enjoy an expansion of that market from competition among low-margin or loss-leading innovators.
“inflammatory bowel disease linked to gene”
I don’t think it’s actually leisure time. It’s a lot of non-productive required micro-tasks that like trying to find a place to live and scrolling an app for good prices on necessities, and mindlessly clicking for information about how to participate in the economy where job skills expire every few months. And watch children and exhaust yourself trying to figure out how to set them up for success and keep them out of trouble. And look for a partner, and do all the things required to compete with extreme selectivity, or extreme self-selectivity.
I’m not wired for all of that. I just want to work.
People will start looking more at your record than your potential, which is a shift from optimism to criticism in 99.9% of cases. In my case, I had what felt like a meteoric rise in the early part of my career, but technology changed, and my interests changed, and now all of that is irrelevant. So all that success you feel like you missed out on so far, it’s probably not much different either way, but you can’t replay it the same way. Maybe you’re talking about money, and lots of things can reset that too.
That big tech co might take you, but increasingly for niche jobs not on the fast track. You can probably do a startup, but you might have to grind on your own for years while some kid gets his tweet funded. When you were young it seemed like the other way around because you only saw the top 0.01% of timelines ahead of you, and now it’s the same way looking back.
Sound advice is the same, but you’ll keep learning that you should have followed it, and there was still time then, but now it’s too late, at least until now becomes then and there was still time. Maybe it sounds something like this: focus on the routine physical act of doing something you respect and enjoy enough to ignore the outcome, and just keep doing it. Or maybe just save your money. Actually don’t ask me; I don’t even know why I’m writing this.
Okay, I do know why: because every other response is going to be some variant of suggestion to pretend that age doesn’t matter, and that your anxiety is totally unfounded. No. Things change. Opportunity is limited. Some doors are closing on you, and some you’ll learn were never open, but it sure feels good to think it’s all possible, and that’s slipping away. I regard dismissal of someone’s feelings as a nasty thing to do to someone, even if you’re just trying to help them, and even if you are rationally correcting them. You’re not wrong, but you’ll adapt and whether by rationality or necessity, correct your values to align with reality as it evolves.
The reality is that the former student will probably never get a place at the table because they’ll get filtered out of the ‘elite’ pipeline sooner than later, and even if they do make it through, they probably also lack the same resources for your companies’s interview process.
Some people want to do it on their own and refuse to cheat. What is the best strategy for someone in that group? Is school and big tech just not for them?
Why say this? Because I’ve learned that the mendacity of paid experts and journalists goes unchecked when ordinary people say nothing, and quite often the crowd opinion of people with nothing to gain is more trustworthy.
Weil is most often encountered in the field of elliptic curves, which they alluded to by the form of the polynomial, but they are mixing the reference to two types of polynomials. The equivalence of finite fields and polynomials was already established by Galois, and is maybe the central theory of abstract algebra. Weil connected this to algebraic geometry through ‘torsion points’ on an elliptic curve, which is a specific form of polynomial itself defined on a (finite) field, that have degree (adding a point to itself until it equals itself again) equal to a prime power in elliptic algebra (addition is the inverse of the intersection of a secant defined by the two points with the curve). This defines a group of prime order, and to get a field, he defined a ‘pairing’ function that works like multiplication between two group elements and returns an element in an ‘extended’ curve that belongs to a group of the same order and has the same coefficient as if the two original group coefficients were multiplied, which is known as the bilinearity property. The extension here is I think what they mean by ‘complex’ in that it doubles the degree of the field polynomial, which gives a Cartesian product of groups from the original definition, and this can be used to divide through ‘torsion points’ back to the field element that defines the product point. This opens up an elliptic form of the zeta function, which was used to prove the Riemann hypothesis on elliptic curves, although not yet extended to the integers, and Fermat’s last theorem. The most common use of this is probably pairing-based cryptography, which is based on bilinearity and computational asymmetry of the pairing operation, which is to say that you can ‘multiply’ through the pairing operation, but you cannot efficiently factor.
So now you have some context of why it’s relevant and some basic terms to look up and you’ll probably find a lot more precise but convoluted language about these things to correct me on.
I have a weakly-held theory that Overtraining Syndrome may be a neurological autoimmune disorder.
I have some personal experience from a very long time ago at a sub-elite level with this as I was diagnosed with the classic ectopic arrhythmia sign where several doctors told me to stop ALL training, and I know of at least one other case that caused the retirement of a reigning world champion. The muscle feeling for me was that training results in numbness, no longer soreness, which was weird and felt like under-training. I do think that poor sleep and anemia were primary factors just as the athlete in the podcast.
To explain, I will go into some uncommon detail about training and recovery. Training starts with adrenaline, and results in a lot of protein breakdown. Recovery begins with the immune system picking up these proteins and releasing prostglandins and other hormones to create inflammation and suppress macrophages from the immune system. Without sufficient suppression, you get damage, most of which can be reversed quickly with local satellite cells, but nerves are harder to repair, which is why you see long-term nervous system manifestations of most autoimmune disorders. Also note the NSAID link. This is theory connecting some dots, but the feeling is where it started. Hard to explain: difficulty engaging strength, burning pain and tingling etc.
I also want to make a different point about the training cycle that they mention. The pull-down pop-up phenomenon, which is the cycle of training, then rest, then higher performance, occurs over a few days, and is almost entirely due to increasing neurotransmitter levels. In simple terms, you are teaching your body to go harder, but the actual muscles aren't changing much. Actual muscular adaptation requires much longer timeframes. If you are just starting any exercise, your fist improvement is your form, then your adrenaline, then your glycogen capacity, then your insulin sensitivity, then your lung efficiency, then your heart vascularization, then your sweat glands, then your bodyweight, then your muscle fiber density, then your muscle fiber quantity. That timeline is somewhere from years to decades, and only the inflammation and adrenaline works on that short-term cycle. Put another way, your physical capacity is not greatly improved after one week of training-recovery cycle. It's actually surprising how little of each season's improvement actually survives a month off, but yet how much a decade of elite training survives years of rest. This is controversial, but if your goal doesn't require near-max performance, then you can ignore the short-term cycle. For instance, to lose weight, it's best not to use this cycle because recovery is much longer in caloric deficit, and you can burn many more calories with low intensity every day rather than high intensity with rest days. This gets more nuanced with goals from other points in the progress timeline.
Another point I'd like to make related to the above, and the podcast. A lot of top-level coaches really suck at training athletes. Even at the olympic level, coaching has a lot more to do with organization, motivation, maybe manipulation, and ultimately selection. That's a separate skill set, and you can make a pretty good team by pushing a large group to their limits until only the best remain. It's not the same thing as getting the best performance out of each individual, which is why it sounds so crazy from her perspective. A top athlete is likely to seek out an independent performance coach after making it through the olympic program filter, and I think this has a lot to do with why the same people keep coming back. They're past the filter and training for performance, not constantly being tested. I think the worst sport for bad coaching might be swimming: endless long sets all season with distance and sprinters mixed into the same lane and then a month of restful performance training at the end. Jason Lezak famously swam the fastest 100 leg in history well past his prime by training himself alone mostly with 50s on a minute. Okay that's far enough off the topic. Peace.