big-chungus4

I suspect the model was just hitting 32k context limit because it is trained for reasoning with 256k tokens available, not 32k

Well, it's better than "X had died" which pops up about once a day and usually contains someone unrelated to hacker news... While this one is inspiring and good to have on HN

Can this produce a useful model? So far 1 bit quants have been less useful than smaller models that use the same memory

My 1k parameter RNN is faster

> W

W what?

It goes the opposite way too. Small models need minimal harness like pi or they get overwhelmed by long system message and tool descriptions

How does he know that this attack is performed by OpenAI agents? I couldn't figure this out from the article

I think it's that when some code compiles to say assembler, the compiler doesn't prioritize readability and maintainability of the assembler code, since people are not expected to read and maintain it directly

What are people on about, there is tons to do in NMS, tons of missions, expeditions, build bases, build a corvet, hunt rate parts, ships, rifles, creatures, manage a settlement, automate crafting chains, find a lucrative trading route, and this new update apps even more things. I'd never consider NMS anything close to empty, there is so much variety in the content

That game is great to RL on to teach the model computer use, they might've actually done that for Astra

Well, do you have any ideas about how to make it scientific? And they clearly needed to do something based on their unnatural ranking

DeepSeek has been the most used model on open router by far, only temporarily overtaken by ox alpha when it was free, so it's the most representative experience

They are great base models for fine tuning on both text and visual tasks. Many OCR and object grounding models are based on small Qwen models, though they often replace vision encoder with a bigger one.

Qwen3 5-4B is the biggest model I can find tune in my laptop. And when I upgraded the model from Qwen3-4B to Qwen3.5-4B, both vanilla and fine tuned performances jumped significantly on a classification task.

Those models are great when you have very little data or very low diversity of examples, where it's not possible to train a neural net from scratch as it will just memorize the data. The best you can do is fine tune a generalist model that can already do the task for small number of steps until it starts over-fitting, or on some cases you can do even better though RL.

> We are releasing these architectural improvements ahead of time so that the community can prepare for the upcoming full family of Qwen4 models.

That gives me hope that "full family" means it will include smaller models like 4B.

I hope there is going to be a free endpoint... Unlike 35B-A3B, I am nowhere close to running it locally

I saw some guy streaming how he was deploying qwen3.8 37B on his local setup. Well, he was asking Claude to do it. It took him two hours of passing errors to Claude for the endpoint to start working, he then started testing it against DS4 Flash when Qwen had thinking disabled and Claude messed up sampling parameters, it was an absolute pain to watch

MCP can also handle authorization, since you don't want to put your password to skills.md and send it to China

So flash is 52 points on artificial analysis, and pro is 53

I was recently working on an AI harness too, but I wasn't using AI to code it. It's really easy and requires little code. I wasn't even using langchain - that would require even less code.

UI is harder for sure, but it's not that bad. You need to think though where to catch which exceptions.

LLMs might opt for langchain which has had multiple breaking changes after the knowledge cutoff, making it hard for the LLM to work with it. This is probably going to lead to the LLM having to make many changes to it's code, making it messy and leading to further code being less maintainable.

It was but I'm not sure what in my reply indicates missing it. The comment still made a point and I replied to it

None of that requires you to be a good public speaker or speech writer

Why exactly is that bad?

Qwen3.7 pro is meh, but 3.7 max is a very good model

I'd say it's going to be very hard to come up with a method that works on general nonconvex functions while not working on convex functions

I'm not sure if the page is not fully loaded for me, because there are a lot of gaps in it, but from what I can see, I still have no idea what "Ode" is. The entire page is just empty marketing without saying anything.

Adam doesn't use the second derivatives matrix, it uses second moments of the gradient, which is the diagonal of the uncentered covariance matrix, but neither of them are directly related to SVD or singular values anyway.

There is a slight connection where Adam approximates full-matrix Adagrad which computes inverse square root of the convariance matrix, which you usually do using eigendecomposition, but on the covariance matrix SVD and eigendecomposition are equivalent (can easily be converted to each other), so you could use SVD to compute the inverse square root.

Last statement is a bit sus... Muon computes matrix sign function which can be defined as setting singular values to 1, though you can also define it without SVD. Muon itself doesn't use SVD because it uses a faster method to compute matrix sign. Adam doesn't do anything related to SVD or singular values. Also not sure what you meant by "second order singular values"

I am sure they also use Google search and Google maps, and it gives them an even greater advantage

A better solution might be not to constrain the generation, but to remove invalid fields from the tool call in the assistant message. So on the next turn, the model receives chat history which contains it's tool call, but without extra arguments. You can do that in OpenAI chat completions / responses, not sure about Anthopic API.

There is still a downside, sometimes the model really wants to include an additional argument for whatever reason it reasoned towards, and it needs the error message to understand that the argument doesn't exist. Otherwise if the argument is manually removed from it's tool call, the model will think that it accidentally left out the argument and start retrying and might go into a loop.

You can take the output of the matrix LSTM, which is going to be matrix for each token, and compute the SVD. To get better storage, we want U and V to be the same for all tokens, so that we can operate on the diagonal S matrix. But LSTM is likely highly nonlinear, U and V will be vastly different for different tokens.