kumama
kumama on Hacker News
i wonder whether these were in the training set for the models
Posted by
kumama on Tuesday October 06, 2026 @01:07PM
from the castform.com dept.
Posted by
kumama on Monday August 17, 2026 @05:25PM
from the castform.com dept.
Posted by
kumama on Friday August 14, 2026 @01:54PM
from the castform.com dept.
cheaper in terms of token costs
castform founder here. unfortunately, we are cloud-hosted at this point.
but some easy options on the open-source side include huggingface's trl & unsloth.
you can run our data-generation scripts here: https://github.com/castform-ai/benchmax and then hook it up trl/unsloth for training. should be able to do all of this on your own compute.
yup! it’s mostly about getting better at using the right search keywords.
for more complex multi-hop question, it's also about knowing which sections of a document to look up and in what order.
+1 on SID-1. we were definitely inspired by that paper
very useful questions here!
- if you have teams that have totally different documentation patterns, yes i think a model trained one teams' docs probably won't directly transfer to another. but let's say you have a model trained on SEC 10-K filings, it should work well as new 10-K filings are added (since terminology & structure of these docs are similar)
- on TCO, the training costs for the experiments you saw were <$200
- you bring up a good point on data drift, don't have concrete numbers/experiments here. something we should defn look into (though i imagine this also changes a lot based on use-case and company)
castform founder here. i'm personally a little against techniques like self-consistency/majority voting during rl training because they tend to result in the model's output distribution "sharpening" a lot. this means the model will lose it's exploration ability and probably won't be able to explore/discover new solution strategies, which can be harmful for both rl training + generalization to unseen cases
castform founder here. i'd say it's a threat but the frontier ai labs' argument would basically be that the market opportunity for intelligence is infinite so it doesn't matter. at the same time, i do believe there will continue to be a massive, growing market for big labs, esp for super-frontier use-cases. today that's longer-horizon coding tasks but in the future it can be things like scientific discovery, etc.
the bigger model would still cost more :)
at the same time, i see prompting as being orthogonal to post-training. i'd imagine post-training a smaller model with a better prompt would make it perform even better
(founder of castform here) the model you post-train should ideally learn general patterns & search strategies over your dataset that should transfer to new docs you add to the search corpus (unless its super out of distribution)
for the example here the chunking is section aware -> but the general training data synthesis pipeline is agnostic to type of chunking
castform founder here. the roi optimization makes sense. i think there are lots of usecases for which even a 2% gain in accuracy can be quite useful. off the top of my head
- high volume customer support. higher accuracy means fewer escalation, reducing labor costs
- fraud detection. catching even one extra fraud attempt could mean a lot in savings
- and ofc the classic ads use-case where at scale bps in improvement could mean millions in revenue :)
this is really cool. i'm sure some of the larger e-commerce companies are already using clickstream data to tune better query rewriters/ranking models
castform founder here. openai actually deprecated their finetuning apis a few months back weirdly.
castform founder here. that's a future we are really excited about too :) ideally, you can post-train the llm within the application itself, as it's being used. both interesting infrastructure & algorithmic challenges here
castform founder here. despite our bet on fine-tuned smaller open-source models, i'm still quite bullish on the big labs. i think scaled closed models will continue to dominate for more general purpose use-case like codegen, search, etc. but intelligence has lots of long-tail applications and i think for these longer-tail applications, finetuned custom models will rule
castform founder here: totally! we also think model routing is also a post-training problem i.e. getting a model to predict the difficulty of a task and match it to the right model -> we're gonna be sharing more on that soon :)
(one of the blog post authors here) -> once you set up a finetuning pipeline, it's often trivial to rerun it on top of a new open weights model. so, it's orthogonal to base model improvements
Posted by
kumama on Thursday July 16, 2026 @03:06PM
from the castform.com dept.
Posted by
kumama on Tuesday June 23, 2026 @04:19PM
from the castform.com dept.
Posted by
kumama on Wednesday June 17, 2026 @02:30PM
from the castform.com dept.