News for hackers. Stuff that matters.

A comment on Launch HN: Outerport (YC S24) – Instant hot-swapping for AI model weights. Parent comment

Re:Launch HN: Outerport (YC S24) – Instant hot-swapping for ... by samstave on Wednesday August 21, 2024 @06:24PM (#41314793)
Could you craft a model-weight as a preamble to a prompt? So you can submit prompts through a layer which will pre-warm the model weights for you based on the prompt - Taking the output into some next step in your workflow, apply a new weight preamble depending on what the next phase is?

Like, for a particular portion of the workflow - assume some crawler of weird Insurance Claims data of scale - and you want particular weights for the aspects of certain logic that youre running to search for fraud.

[ Parent | on HN ]
Re:Launch HN: Outerport (YC S24) – Instant hot-swapping for ... by tovacinni on Wednesday August 21, 2024 @06:40PM (#41314892)
That's a super neat idea- we should in fact be able to use this same system to support the orchestration of a 'system prompt caching' sort of thing (across deployments). I'll put this on my 'things to hack on' list :)
[ Parent | on HN ]