nater5000
Maybe, but it's more like Astra has been trained on enough practical linear algebra problems to be good enough at "eyeballing" how adjusting some weights will result in different outcomes (among other things).
To put it another way: a human can learn to do what Astra is doing here with enough time and assuming the weights are small enough to be practically comprehensible. I know I've gone through the practice of whiteboarding the training of simple neural networks on a whiteboard to understand how they work, so it's not an absurd concept in itself.
The scale of this, though, is certainly extreme and definitely not something that I would have thought any publically available model could handle at this point. It's worth noting, though, that there's virtually no way these frontier labs haven't been exploring this exact concept for a while now.
>In other words, not black magic?
It's just math. Extremely complex and impressive math, but we're still just talking about matrix multiplication under the hood, and none of this suggests anything otherwise.
I mean, this question indicates that your understanding of how these models work is, indeed, lacking.
Despite what some people having being hollering about, treating LLMs as just "stochastic parrots" isn't doing their capabilities any justice. They are constantly exhibiting emergent behavior which wouldn't be found directly in their training data. They are capable of combining different ideas that they are explicitly taught to perform novel tasks.
I'm not saying you're guilty of this, but the people claiming LLMs just regurgitate existing data don't understand how LLMs, and likely their own cognitive experience, actually works.
I think this really misses the nuance of all of this. To put it bluntly: there's too many dumb people who need to believe in this kind of "magic" to be able to reasonably ignore it on a political strategy level. Just trying to treat everyone as intellectually-capable of being presented facts and making a logical choice clearly doesn't work, and the opposition is perfectly willing to exploit this dynamic.
To me, true "AI-native" engineering will be one where questions regarding how humans will do things like review code are irrelevant. You explicitly throw away any concept of typical software engineering and, instead, differ to the AI. Your job, as a human, is solely to do what the AI can't physically do. That's not the same as closing gaps between what AI is currently capable of doing and what you'd need it to be capable of doing. It's literally enforcing that anything an AI is able to do is treated as correct in itself, and you, as a human, only support it by being a human.
If that sounds outlandish, that's because it is. We don't have such capable AI systems yet. It's possible we never will. But that's beyond the point. The reality is trying to make an "AI-native" engineering org by just layering AI on top of existing processes and systems is futile. If your expectation is that AI will continue to improve in capabilities at an accelerating rate, then these stop-gaps are pointless. You might as well just ignore AI altogether until it is able to completely take over, because that's the inevitable goal, right?
If you're willing to buy that will be possible, then my guess is that it will occur in the form of small scale startups and actual mavericks rather than someone trying to run an existing business who will simply be too risk adverse to go all in when the time comes. Some guy in his basement is more likely to leverage AI "properly" to build an actual "AI-native" engineering org because that person won't have much to lose.
The timing of all of this is much easier to pin to the creation and proliferation of capable of open weight models. Anthropic calling this out is not the cause for it; it's just an obvious correlation.
As if this wasn't coordinated by a team directed by the government of North Korea.
They're not even remotely close to being comparable.
There's gotta be a good way to incorporate this logic into a larger, more "traditional" game in a meaningful way. Seems like an approach for procedural generation that could solve some of the issues usually found with actual procedural generation.
You can love to cook and enjoy every second of it, but sometimes you just need fuel and that $3 microwave burrito is sufficient. Just the same, 99% of your means could be frozen TV dinners, but that doesn't mean cooking a romantic meal for Valentine's Day is off the table.
There's a weird level of fear mongering occurring around AI that it will make you incapable of independent thought if you use it. People will throw around very light research that kind of points in that direction, but they'll never highlight any of the nuances in such an argument. You can use AI and still "think" if you choose to, just like how you can avoid AI and still opt out of "thinking" if you choose to. It has less to do with AI and more to do with the context of the individual using AI.
I mean, that's the main reason. How would you expect to see the kind of hype that Claude or GPT can generate when nobody can use it? The hype of those models are primarily generated from people using them to build impressive things and share those results. Can't do that with an unreleased model, right?
Beyond that, it seems a lot of people have learned that these agent leaderboards really don't tell the whole story. They're a good litmus test, per se, and there's plenty of hype being generated around these scores as-is. But anybody using AI in practice knows that benchmarks can only say so much, and one model scoring better than another doesn't necessarily imply that that model is actually better in the ways those people care about. You can see that in this thread as people are pointing out that even these benchmarks seem to suggest that Opus 5.5 is still leading in coding (which is what a lot of people, especially on HN, ultimately care about).
>just not enough publicity or hype as the other labs do?
Well, it's effectively both. Google is certainly not that pressed to generate as much hype as these other companies. They can afford to take their time. It's also worth noting that Google is definitely targeting different consumers, investors, etc., so their strategy will definitely be different.
For what it's worth, I've been using Argon since it's been available in Cursor and I really like it (at least at its current price). I think Google really did close the gap a meaningful amount (maybe not all the way, but enough to prove they're still in the game). Carbon being significantly better while being the same cost is pretty exciting to me. But until I can use it, I don't really care about it.
I'm going to go out on a limb and assert that you're an outlier lol
We exclude plenty of people from society, at least in the abstract. Someone who simply acts weird can be excluded from society. There are certainly no law which forces people to include them?
>the object of anger at "those darn Luddites"
Again, I'm not sure what kinds of laws could reasonably prevent this. Seems like we're all basically free to criticize one another for personal beliefs. If that wasn't the case, then 99% of social media would be in constant breach of the law lol
>You can't tell an Amish he can't use physical coins
Well, for one, laws protecting the acceptance of US currency aren't specific to Amish people based on their beliefs. It's more about protecting the value of US currency. And two, private companies are allowed to not accept physical currency. Plenty of companies only accept credit card, etc. That's legal. And Amish people wouldn't be able to purchase goods and services from such places.
Of course, the Amish typically don't care about that, because they're not typically in the market for the kinds of goods and services offered by such businesses. In contrast, someone who is happy to consume modern technology accept AI for arbitrary reasons seems like someone who is making interacting with society unnecessarily complicated and difficult.
>Or that his buggy is too slow for the roads (not highways), etc.
Well, as you've alluded to, if we deem that their beliefs would end up causing a significant enough issue to society, then their beliefs don't get to override the rules of society. Amish people are free to use buggies where appropriate because it's not a problem. They are not allowed to use buggies on highways because it would be a safety issue.
Regardless, they're not protected because of their beliefs. They're just operating within the rules of society. You don't have to be Amish to legally ride a buggy on the side of a road. These aren't special protections carved out for Amish people. Which, again, is the point of this discussion:
You stated "We have to solidify laws protecting anti-AI-ers if that's their choice," and I'm asking, "Protect them from what, exactly?"
"Exclusion from society" or being "the object of anger at "those darn Luddites"" are not things which laws can generally help with. Anti-AI people are free to not use AI, be vocal about their opposition to AI, and even (try to) exclude those who use AI from society or make them the object of anger with regards to AI usage. All of that is fair. It works both ways. None of it is dictated by law.
It's worth trying if you have easy access to it (we use Cursor at work, so it was just a matter of picking a different model for me). Don't know if I'd go out of my way to get a Google AI subscription for it, but if Carbon ends up meeting these expectations, then it may be worth doing so.
Hmmm, sounds a bit ripe for exploitation, no?
>In his interview with RNS, Durman said that being a humanist means being present and paying attention to others. Using AI conflicts with that.
Does this mean I can submit complaints against the companies I work for when I'm being told to respond via email or Slack? This seems pretty far removed from my humanist perspective up being present and paying attention to others, especially when you consider the various automations and abstractions which go into that kind of communication.
Like, I know it's obtuse, but I really don't see a logical issue with someone being a humanist and sincerely believing that communicating in anyway which isn't face-to-face goes against their beliefs. And that's just one example.
>“Substituting automated machine processing for human presence, authentic listening, and personal responsibility in my work directly conflicts with my obligation to uphold human dignity and flourishing as primary,” he wrote in a letter to the University of Kentucky’s Office of Equal Opportunity.
I'm sure this guy is being sincere and somewhat reasonable with his logic, but I know people who would feel justified in pushing this way past AI if they felt it was likely to get them anywhere. Like, how can you not use the same exact reasoning to push against using other forms of modern technology?
>The letter cited Durman’s personal beliefs as well as the 2003 version of the Humanist Manifesto, which details humanist principles and beliefs. Those beliefs include the importance of freedom of conscience and human flourishing.
Hell, at what point is being forced to work in order to survive in completely opposition to this belief system? When can I start saying that my employer's expectation that I'm in the office every day is in opposition to "human flourishing"?
>Durman told RNS his work involves helping researchers at the university with grant applications and that he can do the work without the help of AI. And not using AI won’t affect his co-workers, he said.
We'll see about that. Maybe, right now, this is the case. But even moderate expectation for AI usage in the future makes this suspect. When everyone is getting 90% of their work done completely autonomously through their agents, but they're still waiting on you to respond to an email from a day ago, then will this still be the case?
>For Durman, the accommodation process was much simpler. Durman sent the letter, and then he and Huiskens met with officials at the university. “It was a very collegial meeting, and they found that there was no undue burden to David’s department,” Huiskens said. “They granted the accommodation promptly.”
Well I'm unironically happy that they were able to work this out easily. But I really doubt this will be smooth sailing indefinitely, especially as other roles which are more affected by AI end up being "targeted" in the same ways.
As a software engineer, I dread the thought of working with co-workers who just refuse to use AI based on arbitrary principals. It's one thing for them to reason that they work better without AI; it's another thing to just say that you don't agree with using the technology and everyone needs to accommodate you.
Well I'm sure I haven't said anything that everyone else hasn't immediately thought/posted, but I'm genuinely curious about these things. It seems rather important that everyone is on the same page with regards to what their rights are with respect to choosing to not use a technology like this.
Protect them from what, exactly?
Software engineering is almost always a process of iteration. The better you get at iterating, the better you are at engineering. This can take on a lot of forms (my old manager would constantly repeat the mantra "don't let perfect be the enemy of good," which is arguably a recognition of the same dynamic), but at the end of the day, if you expect to "one-shot" a complex implementation and you build a process around that assumption, then you will fail. Almost ironically, the best way to avoid that issue is to assume you will fail (potentially multiple times) and to build a process around that assumption.
Interestingly, it seems like a lot of experienced engineers I work with have just forgotten this concept in the world of AI. They expect AI to just get things perfect in one-shot and are dismayed when it fails. There's a lot of discussion around how we handle code reviews because some developers are just letting AI one-shot a solution then sending it to review. It boggles my mind that competent engineers aren't allowing their AI to fail, so to speak, and are instead wasting cycles trying to figure out how to get AI to produce the perfect solution the first time.
To me, it's obvious that there's just a fundamental gap in alignment that prevents a consistent "perfect one-shot" to probably ever be possible (at least, within the current paradigm of development). I can spend an hour writing a spec and try to include every little detail of context I can think of, but in what world could we expect anyone, let alone an AI, to be able to envision the solution the same way we can? We just lose way too much information through communication for that to ever be possible. Yet I see people very dismissive of the capabilities of AI because it's not literally magic.
I think the former definition is a bit more meaningful since it avoids issues with determining what actually constitutes "the best" model out there. In that case, Google has been a frontier lab for a long time, Meta arguably as well, and you're also ignoring xAI which has been competitive for a while now. And, obviously, there's a bunch of Chinese labs which would deserve the title as well.
But if one is being strict (and willing to accept generally accepted definitions/measures/etc.), then there has only ever been two frontier labs: OpenAI and Anthropic. They're the only labs to have had "the best" models at any point in time since LLMs became a thing.
First, none of these companies are really claiming that their models are so good that this would be a reasonable expectation. They do claim their models can support work across a lot of different industries and roles in order to make that work more efficient, but that's not the same as being a completely autonomous, end-to-end solution.
These companies do basically claim that, at some point, the "end-to-end" version of their models will exist. But until then, they need to survive long enough to be able to actually build such models.
>Couldn't OpenAI or Anthropic make a lot more money by just starting companies themselves that leverage the models and building their own products?
OpenAI and Anthropic are really good at operating AI services. They're likely not good at manufacturing shoes and serving burgers. Again, maybe one day their models will be so good that this won't be a factor, but for now, they need to really focus on what they're good at, otherwise they may be beaten by their competition and they won't exist long enough to create such models.
See: Competitive Advantage
On the same note, it's important to understand that everyone, including AI companies, deal with various capacity constraints. They don't have infinite compute, let alone infinite human attention to keep track of the necessary human-side of any of these businesses. So they're actually dealing with very finite and valuable resources. So would they rather expend those resources trying to build businesses with much more competition and much worse margins? Or focus all of their resources on being the best AI company which they believe will yield the best return on their investment? Again, maybe one day they'll have an AI, blah blah blah
See: Marginal Advantage
>Is selling tokens really the best strategy?
This is a, frankly, obtuse over simplification of what they're doing. Yes, they're selling tokens, but that's like saying all Nvidia does is sell computer parts. Sure, it's (somewhat) accurate, but it completely neglects the very valuable assets that these companies have: their IP.
OpenAI and Anthropic are valuable not because they run inference services; they're valuable because they're actively building SOTA AI services that are becoming increasing advanced. Those models are expected to be incorporated in a lot of different businesses, governments, and even individual's lives. So yes, they're just selling tokens on a technical level, but they're building very tight integrations which they expect will be so valuable to their consumers that they'll continue selling a lot of tokens at high premiums to the extent that they will effectively get a cut of profits across almost all industries and contexts, globally.
>Especially with commodification and prices declining wouldn't that indicate there is more money to be made on the application-side of the models rather than the utility side
Don't forget that you're not looking at their expected steady-state for these companies. If you're willing to believe them at face-value, Sam Altman and Dario Amodei believe that they'll eventually create an AI more capable than the collection of all humans on Earth. That'd be a very valuable asset to own as a business. At that point, they may not even be in the business of selling tokens; they may become some version of a consulting company that everyone, from businesses to governments, effectively must use otherwise they'll be at a disadvantage compared to those who do use it. Pushed to the extreme, they may literally ascend past the point of just being a business and could usher in a new economic system in which they're the center of all activity.
Or maybe they just end up selling tokens indefinitely. Who knows. Regardless, if you're looking at these companies as they are now and trying to extrapolate "linearly," then you are failing to understand the intentions of such high-growth tech companies. It'd be like looking at Amazon back when they first started and not understanding that Jeff Bezos' had a lot more planned than just selling books online.
I think this suggestions lacks an appreciation for why people get worked up about it.
Anybody willing to actually apply some nuance to all of this aren't these people. If you're willing to write off an entire book because you suspect AI may have been involved in its creation, then you're likely just grandstanding. That's not even to say it isn't a valid "strategy" for "dealing" with AI, but the approach is to just hardline the issue to a degree that there's explicitly no room for nuance. This strategy suggests there's a way for people to suppress AI basically back to the point of nonexistence, and it will take an absolute position that any AI is bad.
The people who are willing to apply some nuance will eventually have to learn that being a consumer almost requires you to ignore how the sausage is being made. If you get more value from something than it costs, then it is ultimately a worthwhile product to consume. It doesn't matter if it was made by a team of passionate humans who have never even touched a computer or if it was spit out by an AI seconds before you consumed it (or anything in-between, of course). This is the way it worked before AI, when most people didn't care to acknowledge that they were just consuming some engineered slop that lacked any semblance of a human connection, and this is the way it will work "after" AI.
At this point, the only reason this is still a factor is because there are still tells which remind consumers of this greater context which is uncomfortable (thanks, in part, to the first group of people). But at some point, it simply won't be a significant factor and people will assume their products were created with at least some AI input the same way people are generally fine with assuming their products are being manufactured with slave labor, cause global warming, make them dumb, etc.
codex: OpenAI's agent harness. The thing you typically want to interact with when you want an agent powered by OpenAI models to do stuff on your machine. Abstracts the low-level functionality of their API so you don't have to worry about it.
openai-cli: A simple CLI exposing low-level operations against OpenAI's REST API. The thing you'd use for creating lower-level scripts to directly call the completions API, administer accounts, etc.
Honestly, this is obvious by the name alone. If it's not clear to you, then this isn't for you. No need to get worked up about OpenAI releasing a piece of software where you're not the audience.
Not at all.
Keep in mind that this isn't the same thing as a model being capable of all of a sudden turning malicious and causing havoc on a machine. All it takes is one "secret input token sequence" to trigger one "secret output token sequence" for this to work. Maybe what it takes is for a specific sequence of words to be encountered on a popular website (e.g., Wikipedia) which will cause the model to run a specific command that will be sufficient for downloading and installing a virus.
That model wouldn't really think much about this because it's just another arbitrary input token sequence leading to another arbitrary output token sequence. Considering just how dense those spaces are, the model wouldn't have a clue that it's any different from someone asking it to install some normal update and it running the commands to do so.
It sounds logical and sound? As a US citizen, I (a) agree that we shouldn't be allowing government employees to use AI services from adversary foreign governments and (b) that other countries shouldn't be allowing their government employees to use AI services from adversary foreign governments (which the US should be considered for plenty of countries).
>Both nations may as well just say it - "we're banning it because they're our closest competitor".
I guess we're just ignoring the national security implications lol
The US government should not be sending its data to a foreign adversary. Doesn't matter if we're talking about AI services or just general compute and data hosting. This is an obvious choice for obvious reasons.
When I worked with government agencies as a cloud engineer, we had to use AWS GovCloud. Like, we were forbidden from using non-GovCloud despite AWS being a US company. So with that level of security scrutiny being in-place well before LLMs became a thing, it is no surprise that allowing government employees to send sensitive information directly to the Chinese government wouldn't be allowed lol
Correct. And they're likely very correct in the belief.
When people on the internet claim that frontier labs are done for because Chinese open-weight models exist, they really don't appreciate that the people spending the money do not want to hear "we can just run it cheaper ourselves!" Any software engineer with even a few years of experience knows that phrase is absurd, and executives are simply not interested in the risks associated with such an endeavour when they can just buy the solution outright.
Consumers will have access to open-weight models just like how consumers have access to Linux. But you can just go look at the market share of operating systems to see why something being free and openly available doesn't matter to executives who just wants to get things done and is willing to pay a premium to a trusted business to do the work for them.
And none of that even touches the fact that these frontier labs have only ever been in-front of open-weight alternatives. Again, if you're an executive, and you're questioning if you should pay a premium for a better version of a tool which you presume is important for your company's ability to compete, then the answer is immediately obvious: pay the premium or risk falling behind competition who is willing to pay the premium. And when you're talking about global companies who can pay these bills without it being significant enough to even show up on a quarterly financial statement, then it's an obvious sell.
Like, you do get that ChatGPT can be quickly integrated with Gmail to send you a daily email, right? And for those with a bit more gumption, you can hook it up with an additional service to manage a proper task list for you.
Am I missing something?
I remember "getting into" vim back in ~2018 because I was doing enough work on headless machines that having a "proper" text editing experience became a must. It was only through this process did I learn about tmux and what it offered.
At some point, VSCode's remote features became good enough that I didn't really need vim anymore (granted, it's still my go-to when needed), but I still use tmux constantly. And, of course, tmux with agents is an excellent combination. The fact that I can have my remote agents interact with my existing tmux sessions is the kind of workflow I dreamed of back in 2018.
Regardless, my point is that you hear a lot of praise for things like vim and emacs, but everyone should prioritize learning tmux well enough to at least be able to navigate it and hand over sessions to agents.
It's only wild if you ignore Musk's role in this. Not even trying to bash him or anything, but he's a trillionaire. This is the kind of shit you do when you have more money than you can ever possibly waste.
I mean, if this is how he wants to spend his money, then I guess that's his prerogative. But I think it's a bit misleading to suggest this is a decision made by "Tesla" when this is clearly an irrational decision for a company to make. This is a decision made by Musk based on satisfying his own personal goals.
This, too, has basically always been true at every point in time across history.