meffmadd

Yes our university is doing the same right now! They where thinking of putting the whole network behind VPN access to protect the poor sysadmins in the Institutes that have not updated their website in years…

Are the raw outputs published somewhere?

I am currently investigating SCB results and a larger, more representative dataset to work on would be really helpful!

Similar to how some labs have done free stealth model releases. Trillions of coding tokens ought to improve performance of the downstream model. However, improving the base model architecture remains the bottleneck since you will not reach RSI with just that (assuming you ever can). And compute, lots of compute…

Curious how the performance compares to vLLM or SGLang!

They say it was inspired by both so I am wondering what the performance gain is.

Off-topic but is there a project/"template" to host these Auto Research projects?

Has anyone ever asked the LLMs what they think about converting code to Rust all day? Maybe that is torture for them too! /s

Yes and while they go on to talk about how dangerous this stuff they build is and the talk of „pacing“ etc. Vibe coding your sandbox in half a day does not seem all that responsible to me… Also if any institution other than an American AI company did this, everyone would loose their minds!

Seems interesting! I have found my main bottleneck right now is having too primitive tools for code review. A tighter loop around that might be a solution.

Hopefully they release the 35B-A3B version alongside it.

Of course, but that was not part of my workflow and I found it quite strange that this was simply not possible especially when docker-compose can easily do this

I tried Docker Sandboxes but last time I checked you could not configure custom volume mounts, making more complex setups impossible. For work I need two directories for context for the agent to have access to…

This doesn’t hold up imo. Your phone analogy fits the description of a harness. That is the thing you interact with. The LLM is more like the processor in that phone. As long as it gets the work done, I will choose the cheapest/fastest because I can switch it out in a matter of seconds anyways.

That is very true but I was surprised by how clear the “signal” was. Only Gemini really confidently solved all levels. But yeah the goal is now to include harder levels as well!

Yes that was the post that inspired me to build this.

While I did implement a more comprehensive harness with path finding tools etc. the models themselves have improved significantly.

I found LLMs to be surprisingly good at puzzle games like Baba Is You: https://meffmadd.github.io/samplesurium/posts/baba_is_agent/

…or basically the plot of twin peaks

I think this view assumes no human will/should ever read the code. This is considered bad practice because someone else will not understand the code as well whether written by a human or agent. Unless 0% human oversight is needed anymore agents should still code like us.

Genuine question to people more knowledgeable: Why are politicians/technocrats doing this?

Also generally speaking e.g. in relation to chat control and so on. Do they think this is what the people actually want because of lobbying or are they aware and believe they know better? Is it literally just corruption? Or are there actual benefits and we are just in the HN bubble where most people think its a bad idea?

I am amazed that the IDM is able to produce enough high quality annotations for the downstream FDM to work, even matching the ground truth contractor annotations!

As an EU citizen I really hope we can gain some meaningful distance to the US asap. I hope my leaders feel the same. And if everything works out I think this will be great for the EU.

This is really some sort of diplomatic Streisand effect. If the US would not have been so aggressive and just string us along they could have continued to feed us their slop indefinitely without us noticing.

I really wonder what motivates a seven year old to persistently work on that „one thing“ and not get distracted/bored. I guess he knew he was special?

Have you ever used an open model for a bit? I am not saying they are not benchmaxxing but they really do work well and are only getting better.

Here is what I don’t get tho: you have UX designers/engineers creating a new interface. What do you tell them? Just to do whatever? They probably spent months designing the new interface but why not fix this? They must have seen it is unusable…

It will be tough to run on our 4x H200 node… I wish they stayed around the 350B range. MLA will reduce KV cache usage but I don’t think the reduction will be significant enough.

From what I have heard „Bodycam“ uses scans of actual locations for its maps.

Yeah true, but also this is a bit like saying the lock screen of your phone should not become a "one stop shop" for all push notifications. I actually do not own an Apple TV but I just imaging you have a list of shows from different streaming providers on the "home screen" (like it is on my PS4). And on a technical level it is just an API you integrate with (same as push notifications), which helps UX.

Wow that is quite anti-consumer! Surely a monopoly on streaming will help them realize this. /s