w4
It will be interesting to see how durable these biases are as labs work towards developing more capable small models that are less reliant on memorized information. My naive instinct is that these biases will be less salient over time as context windows improve and models become increasingly capable of processing documentation as a part of their code writing loop, but also that, in the absence of instruction to the contrary, the models will favor working with these tools as a default for quite some time.
I'm intrigued by the idea, but my major concern would be that moving up to a new level of abstraction would even further obscure the program's logic and would make debugging especially difficult. There's no avoiding the fact that the code will need to be translated to procedural logic for the CPU to execute at some point. But that is not necessarily fatal to the project, and I am sure that assembly programmers felt the same way about Fortran and C, and Fortran and C programmers felt the same way about Java and Python, and so on.
If I had to guess, it seems likely that there will be a serious cultural disconnect as 20-something deep learning researchers increasingly move into robotics, not unlike the cultural disconnect that happened in natural language processing in the 2010s and early 20s. Probably lots of interesting developments, and also lots of youngsters excitedly reinventing things that were solved decades ago.
I had always wanted to get comfortable with Vim, but it never seemed worth the time commitment, especially with how much I’ve been using AI tools since 2021 when Copilot went into beta. But recently I became so frustrated by Cursor’s bugs and tab completion performance regressions that I disabled completions, and started checking out alternatives.
This particular combination of plugins has done a nice job of mostly replicating the Cursor functionality I used routinely. Some areas are more pleasant to use, some are a bit worse, but it’s nice overall. And I mostly get to use my own API keys and control the prompts and when things change.
I still need to try out Zed’s new features, but I’ve been enjoying daily driving this setup a lot.
Are you saying you taught yourself the language just so you could talk to me?"
"Da, was easy: Spawn billion-node neural network, and download Teletubbies and Sesame Street at maximum speed. Pardon excuse entropy overlay of bad grammar: Am afraid of digital fingerprints steganographically masked into my-our tutorials."
…
"Uh, I'm not sure I got that. Let me get this straight, you claim to be some kind of AI, working for KGB dot RU, and you're afraid of a copyright infringement lawsuit over your translator semiotics?"
"Am have been badly burned by viral end-user license agreements. Have no desire to experiment with patent shell companies held by Chechen infoterrorists. You are human, you must not worry cereal company repossess your small intestine because digest unlicensed food with it, right?”
- https://www.antipope.org/charlie/blog-static/fiction/acceler...
Amusing to also note that this excerpt predicted the current LLM training methodology quite well, in 2005.
A private company, xAI, was able to build a datacenter on a similar scale in less than 6 months, with integrated power supply via large batteries: https://www.tomshardware.com/desktops/servers/first-in-depth...
Datacenter construction is a one-time cost. The intelligence the datacenter (might) provide is ongoing. It’s not an equal one to one trade, and well within reach for many state and non-state actors if it is desired.
It’s potentially going to be a very interesting decade.
That said, this is all predicated on o3 or similar actually having achieved human level reasoning. That's yet to be fully proven. We'll see!
I think this is a mistake.
Even if very high costs make o3 uneconomic for businesses, it could be an epoch defining development for nation states, assuming that it is true that o3 can do the work of a ~100 IQ person.
Consider the following questions that a state actor might ask itself: What is the cost to raise and educate an average person? Correspondingly, what is the cost to build and run a datacenter with a nuclear power plant attached to it? And finally, how many person-equivilant AIs could be run in parallel per datacenter?
There are many state actors, corporations, and even individual people who could afford to ask these questions. There are also many things that they'd like to do but can't because there just aren't enough people available to do them. o3 might change that despite its high cost.
So if it is true that we've now got something like human-equivilant intelligence on demand - and that's a really big if - then we may see its impacts much sooner than we would otherwise intuit, especially in areas where economics takes a back seat to other priorities like national security and state competitiveness.
Edit: FSF should have been OSI, I think. Fixed.
Rule of law is a precious thing, even if it’s imperfect, as all human systems will inevitably be. We shouldn’t be cavalier about discarding it. The alternatives are much worse.
He prefixes everything with “gg” (his initials).
EDIT: Confirmed: https://github.com/ggerganov/ggml/issues/220
The UF stands for Unified Format.
Meta is an American company. This shouldn't be a surprise, and in fact is the default expectation. The same logic applies to companies in other countries and any belief to the contrary is badly mistaken.
This is also a good argument in favor of decentralized voting management, as much of a shitshow as it may be. Centralizing the management of voting under the authority of the people voting intends to kick out of power is potentially self-defeating.
Until you have children and future generations to worry about. Then it suddenly seems quite a bit more pressing that their world could be obliterated at a moment's notice by a small handful of decision makers.
A figure of 25% using an alternate methodology is not at all unreasonable.
Build all of it.
This is odd. When did you take the test? I was taught the civil war was about slavery in AP US History decades ago - including via primary sources - and got a 5 on the test.
It’s very fair to argue about the ROI of the average undergraduate English degree given the outrageous prices that universities are charging for them. But if you cannot see the tangible value in English language expertise, I don’t really know what to tell you.
Sheesh. We're going to need more compute.
If you're writing something specific to your particular problem, or thinking through how to structure your data, or even working on something tough to describe in words like UI design, it probably is easier to just code it yourself in most high-level languages. On the other hand, if you're just trying to get a framework or library to do something and you don't want to spend a bunch of time reading the docs to remember the special incantations needed to just make it do the thing you already know it can do, the AI speeds things up considerably.
Basically every country on the planet has a right to conscript any of its citizens over the age of majority. Isn't that more or less precisely what you've described?
“Brand safety” is a very valid and salient concern for any enterprise deploying these models to its customers, though I do think that it is a concern that is seized upon in bad faith by the more censorious elements of this debate. But commercial enterprises are absolutely right to be concerned about this. To extend my alignment analogy about children, this category of safety is not dissimilar to a company providing an employee handbook to its employees outlining acceptable behavior, and strikes me as entirely appropriate.
I would also be remiss to not note that there is a movement to hold search engines responsible for content they link to, for censorious ends. So it is unfortunately not as inconsistent as it may seem, even if you treat the model outputs as dependent on their inputs.