vrm

good luck running a 2.4T model on any local hardware. it’s not gonna happen. the arrow is to specialized hardware at least for the smartest models

TBH it's intended only for internal use (we don't even publish it as a crate at this point) so I don't particularly mind it being low-key. But I appreciate it!

If you don't need a ton of throughput I think `absurd` (and our Rust derivative `durable`) are very nice options that keep the client side extremely simple. It's also lightweight enough that a coding agent can keep the entire thing in its head easily and just run queries to look up state as needed.

Since DBOS doesn't support Rust, we implemented a very minimal Rust version of this at https://github.com/tensorzero/durable. It has been quite stable and extensible but of course you need to be very careful with the SQL implementations. Hope this is interesting to readers here.

It’s really not a concept you can express in idiomatic Python very easily. This comes from the actual generated assembly involving copies from global GPU memory into registers (slow, bandwidth saturates quickly) and back in between the cosines. If you can avoid the intermediate roundtrip that cuts the cost approximately in half.

One question I have here: I think this type of thing would be trivial to do in Rust with constructors, private fields, and newtypes. What am I getting on top of it?

that is earnings (net income) not revenue (top line) so these are wildly different and incomparable numbers

a 6:1 parameter ratio is too small for specdec to have that much of an effect. You'd really want to see 10:1 or even more for this to start to matter

This is neat! I think in general there are really deep connections between semantically meaningful diffs (across modalities) and supervision of AI models. You might imagine a human-in-the-loop workflow where the human makes edits to a particular generation and then those edits are used as supervision for a future implementation of that thing. We did some related work here: https://www.tensorzero.com/blog/automatically-evaluating-ai-... on the coding use case but I'm interested in all the different approaches to the problem and especially on less structured domains.

if you haven't check out our repo -- it's free, fully self-hosted, production-grade, and designed for precisely this application :)

https://github.com/TensorZero/tensorzero

I definitely see different prompts based on what I'm doing in the app. As we mentioned there are different prompts for if you're asking questions, doing Cmd-K edits, working in the shell, etc. I'd also imagine that they customize the prompt by model (unobserved here, but we can also customize per-model using TensorZero and A/B test).

we're doing the latter! Cursor lets you configure the OpenAI base URL so we were able to have Cursor call Ngrok -> Nginx (for auth) -> TensorZero -> LLMs. We explain in detail in the blog post.

wireshark would work for seeing the requests from the desktop app to Cursor’s servers (which make the actual LLM requests). But if you’re interested in what the actual requests to LLMs look like from Cursor’s servers you have to set something like this up. Plus, this lets us modify the request and A/B test variations!

We're working on an OSS industrial-grade version of this at TensorZero but there's a long way to go. I think the easiest out of the box solution today is probably OpenAI RFT but that's a partial solve with substantial vendor lock-in.

This is very neat work! Will be interested in how they make this sort of thing available to the public but it is clear from some of the results they mention that search + LLM is one path to the production of net-new knowledge from AI systems.

would it be possible to fuzz an arbitrary JSON schema with this? I’ve been looking for such a lib for a while now

OP here: I saw the spinlaunch video and got really excited that this was completely tractable in simplified form with basic physics. So I did the math expecting to see a huge savings in fuel mass and....it was basically negligible. I'm curious if anyone on HN (maybe even from Spinlaunch!) could explain where I go wrong. Otherwise you might find it interesting to read. Thanks!

Yeah so the paper I linked on contextual bayesian optimization (https://papers.nips.cc/paper/2019/hash/7876acb66640bad41f1e1...) does a combination of controlling for beta and optimizing linear MHD stability --- which is part of the problem. One of our collaborators has been working precisely on disruption prediction and mitigation but it's on most of our minds: https://www.pppl.gov/news/2021/artificial-intelligence-helps....

will repost my comment from the other one:

Papers I would recommend from our collaboration on control of normalized plasma pressure: https://papers.nips.cc/paper/2019/hash/7876acb66640bad41f1e1...

plasma profile transport modeling: https://iopscience.iop.org/article/10.1088/1741-4326/abe08d/...

hybrid dynamical modeling of gross plasma quantities: https://arxiv.org/abs/2006.12682

uncertainty quantification for plasma dynamics: https://arxiv.org/abs/2011.09588

It's still early days for this work and for us but we're looking at pushing reinforcement learning in methods and engineering to solve this problem.

I need to redo my website, just getting into the more public part of my PhD. Papers I would recommend from our collaboration on

control of normalized plasma pressure: https://papers.nips.cc/paper/2019/hash/7876acb66640bad41f1e1...

plasma profile transport modeling: https://iopscience.iop.org/article/10.1088/1741-4326/abe08d/...

hybrid dynamical modeling of gross plasma quantities: https://arxiv.org/abs/2006.12682

uncertainty quantification for plasma dynamics: https://arxiv.org/abs/2011.09588

It's still early days for this work and for us but we're looking at pushing reinforcement learning in methods and engineering to solve this problem

like I said above, we certainly hope so! It has been slow progress so far but applying modern ML / control techniques to tokamaks is one of the truly exciting applications of the current generation of AI in my opinion. Biased because this is literally what I do all day