comp_raccoon

Olmo author here! Your are absolutely spot on on

> It was impossible for me to actually fact-check any of the claims in the response based on the matched training data.

this is true! the point of OlmoTrace is to show that even the smallest phrases generated by a langue model are a product of its training data. It’s not verification; a search system doing post hoc checks would be much more effective

Olmo author here… would be nice to have some more competition!! I don’t like that we are so lonely either.

We are competitive with open weights models in general, just a couple points behind best Qwen.

Fully open models are important for research community; a lot of fundamental discoveries are made when you have access to training data. We call out we are the best fully open model because researchers would want to know about that.

Olmo author here, but I can help! First release of Qwen 3 left a lot of performance on the table bc they had some challenges balancing thinking and non-thinking modes. VL series has refreshed posttrain, so they are much better!

Olmo author here! Qwenmodels are in general amazing, but 30B is v fast cuz it’s an MoE. MoEs very much on the roadmap for next Olmo.

Olmo author here! we release all training data and all our training scripts, plus intermediate checkpoints, so you could take a checkpoint, reproduce a few steps on the training data, and check if loss matches.

it’s no cryptography proof, and you can’t get perfect determinism on nvidia GPUs, but it’s pretty close.

Olmo researcher here. The point of OlmoTrace is not no attribute the entire response to one document in the training data—that’s not how language models “acquire” knowledge, and finding a single or few documents as support for an answer is impossible.

The point of OlmoTrace is to show that fragments of model response are influenced by its training data. sometimes is how specific adjectives are used together in way that seem unnatural to us, but are combination of training data (ask for a movie review!)

A favorite example of mine is asking to tell a joke or ask for a random number, because strangely all LLMs return the same joke or number. Well with OlmoTrace, you can see which docs in the training data contain the super common response!

hope this helps

This is correct! we wanted to show that you can use PixMo dataset and our training code to improve any open model, not just ours!

google image APIs are not great, yeah it’s only for demo, though—checkpoints on huggingface are uncensored.

it’s coming! just takes a bit more time to properly release it.

they have a technical report coming! knowing the team, they will do a great job disclosing as much as possible.