reexpressionist

A major limitation of directly using "System 1 decision models" (a.k.a., logistic regression, and related uncalibrated classifiers over the output/logit space) in enterprise settings (or other high-stakes settings) is that such estimators are not reliable estimators of the predictive uncertainty in the presence of covariate shifts, and such estimators also lack a means of instance-wise data attribution (i.e., interpretability-by-exemplar), so they're not the ideal estimator for verification, routing, uncertainty over retrieval and tool-calls, etc.

For decision-making with neural networks, we instead need the older idea of estimators of the predictive uncertainty with constraints in the feature-representation space (over training/support of the estimator), as with Similarity-Distance-Magnitude estimators: https://pypi.org/project/reexpress-sdm/

The key properties for using such models for conditional-branching decisions in agentic stacks (and related) is that they should be well-calibrated (under the definition chosen for the task) and informative (e.g., always predicting the mean might be "well-calibrated" in a theoretical sense for some chosen quantities of interest, but isn't particularly useful in practice).

The tricky thing with the neural networks is that the output logits are in effect a highly lossy compression of the epistemic (reducible) uncertainty, so even if the target calibration quantity is well-specified, it can be difficult to obtain in practice. A side-effect of this is that estimates in the high probability regions are not particularly stable under even modest co-variate shifts, which is a real problem if the estimates are being used for decision-making in a multi-step search graph that can lead to branches that are unlike what the model/estimator saw at training/calibration (if not altogether out-of-distribution). Here are a couple papers that describe how to approach those challenges:

[1] Similarity-Distance-Magnitude Activations. In Findings of the Association for Computational Linguistics: ACL 2026, pages 22037–22057, San Diego, California, United States. Association for Computational Linguistics.

[2] Introspectable, Updatable, and Uncertainty-aware Classification of Language Model Instruction-following. In Proceedings of the ACM Conference on AI and Agentic Systems (CAIS '26). Association for Computing Machinery, New York, NY, USA, 1259--1269.

We may well look back in future years and view the underlying approach introduced in Reexpress as among the more significant results of the first quarter of the 21st century. With Reexpress, we can generate reliable probability estimates over high-dimensional objects (e.g., LLMs), including in the presence of a non-trivial subset of distribution shifts seen in practice. A non-vacuous argument can be made that this solves the alignment/super-alignment problem (the ultimate goal of the line of work in the post above, and why I mention this here), because we can achieve this behavior via composition with networks of arbitrary size.

Because the parameters of the large neural networks are non-identifiable (in the statistical sense), we operate at the unit of analysis of labeled examples/exemplars (i.e., the observable data), with a direct connection between the Training set and the Calibration set.

This has important practical implications. It works with essentially any generative AI model. For example, we can build an 'uncertainty-aware GPT-4' for use in enterprise and professional settings, such as law: https://github.com/ReexpressAI/Example_Data/blob/main/tutori...

(The need for reliable, controllable estimates is critical regardless of any notion of AGI, since the existing LLMs are already getting baked into higher-risk settings, such as medicine, finance, and law.)

This type of behavior (and related) would primarily only be an issue with unconstrained generative models. If you're the one deploying the model, or a downstream consumer, once trained, the neural network can be reexpressed (via an exogenous/secondary model/process) to derive reliable and interpretable uncertainty quantification by conditioning on reference classes (in a held-out Calibration set) formed by the Similarity to Training (depth-matches to training), Distance to Training, and a CDF-based per-class threshold on the output magnitude. If the prediction/output falls below the desired probability threshold, gracefully fail by rejecting the prediction, rather than allowing silent errors to accumulate.

For higher-risk settings, you can always turn the crank to be more conservative (i.e., more stringent parameters and/or requiring a larger sample size in the highest probability and reliability data partition).

For classification tasks, this follows directly. For generative output, this comes into play with the final verification classifier used over the output.

Here's the direct link and I can confirm that Mixtral-8x7B-v0.1 works on M2 Ultra 128GB via MLX and is easy to setup (the longest part is just downloading the weights):

https://github.com/ml-explore/mlx-examples/tree/main/llms/mi...

We'll have a tutorial soon (next week) on combining/composing with Reexpress to add uncertainty quantification (and to use it for semantic search). A link will be here: Tutorial 6: https://re.express/guide.html

On the topic of errors in LLM outputs (and on-device/local LLMs):

You can also reexpress such an LLM model with a classification verifier to get a probability estimate...entirely on your local Mac: https://re.express/

This is a case where the advance/breakthrough actually came to the local setting before the cloud/API services.

I believe we have two rather different settings in mind. My statement assumes the enterprise use-case, where having a verifier is required. (In this context, I'm also assuming the approach of constraining against the observed data.) In such a selective classification setting, the end-user need not be exposed to lower quality outputs, but rather null predictions if the model cascade has been exhausted (i.e., progressively moving to larger models until the probability is acceptable).

Hopefully in 2024 we can get at least one of the benchmarks to move to assessing non-parametric/distribution-free uncertainty for selective classification, reflecting more recent CS/Stats advances that should be used in practice. Working on it.

The alternative approach is to start with a small[er] model, but derive reliable uncertainty estimates, only moving to a larger model if necessary (i.e., if the probability of the predictions is lower than needed for the task).

And I agree that the leaderboards don't currently reflect the quantities of interest typically needed in practice.

If the end goal is document classification and/or semantic search, the Reexpress Fast I model (3.2 billion parameters) is a good choice. The key is that it produces reliable uncertainty estimates (for classification), so you know if you need a larger (or alternative) model. (In fact, an argument can be made that since the other models don't produce such uncertainty estimates, they are not ideal for serious use cases without adding an additional mechanism, such as ensembling with the Reexpress model.)

TL;DR: Reexpress makes it really easy (and inexpensive) to fine-tune a large language model (LLM) for typical document classification tasks. All of the processing happens on your Mac and you also get the indispensable additional advantages of uncertainty quantification, interpretability by example/exemplar, and semantic search capabilities.

Incidentally, a ((Llama Guard) Guard)()

> "A guard for a Llama Guard that adds robust uncertainty quantification and interpretability capabilities to the safety classifier"

...can be easily created by ensembling your fine-tuned Llama Guard with Reexpress: https://re.express/

(The combination of a large fine-tuned decoder classifier with our on-device modeling constraints is, in all seriousness, likely to be quite useful. In higher-risk settings, the deeper you recurse and the more stringent the reexpression constraints, the better.)

Important essay and points. I want to mention that there exist now practical technical approaches that can be used to create trustworthy AI...and such approaches can be run on local models, as this comment suggests.

> "[...] [AI] will act trustworthy, but it will not be trustworthy. We won’t know how they are trained. We won’t know their secret instructions. We won’t know their biases, either accidental or deliberate. [...]"

I agree that this is true with standard deployments of the generative AI models, but we can instead reframe networks as a direct connection between the observed/known data and new predictions, and to tightly constrain predictions against the known labels. In this way, we can have controllable oversight of biases, out-of-distribution errors, and more broadly, a clear relation to the task-specific training data.

That is to say, I believe the concerns in the essay are valid in that they reflect one possible path in the current fork in the road, but it is not inevitable, given the potential of reliable, on-device, personal AI.

In my biased opinion, I'd recommend using Reexpress to get reliable uncertainty quantification from LLMs. The video Tutorial 2 on our website demonstrates how to derive uncertainty estimates from the Mistral 7b model, but the basic principle is the same for any LLM: Simply include the output logits when uploading a document via the JSON lines format: https://re.express/

Ditto. This is the most sophisticated viz of parameters I've seen...and it's also an interactive, step-through tutorial!

"As with most software development, modern AI work is all about knowing your tools and when it's appropriate to use them." 100% agree. Ditto with just using the easiest to access models as initial proof-of-concept/dev/etc. to get started.

(I do agree with the overall sentiment of the TC article, although as noted by others below, there's some mashing of terminology in the article. E.g., I, too, associate GOFAI with symbolic AI and planning.)

There's another dimension, too, not mentioned in the article: Even with general purpose LLMs, for production applications, it's still required to have labeled data to produce uncertainty estimates. (There's a sense in which any well-defined and tested production application is a 'single-task' setting, in it its own way.) One of the reasons on-device/edge AI has gotten so interesting, in my opinion, is that we now know how to derive reliable uncertainty estimates with the neural models (more or less independent of scale). As long as prediction uncertainty is sufficiently low, there's no particular reason to go to a larger model. That can lead to non-trivial cost/resource savings, as well as the other benefits of keeping things on-device.

I like the analogy to a router and local Mixture of Experts; that's basically how I see things going, as well. (Also, agreed that Huggingface has really gone far in making it possible to build such systems across many models.)

There's also another related sense for which we want routing across models for efficiency reasons in the local setting, even for tasks for the same input modalities:

First, attempt prediction on small(er) models, and if the constrained output is not sufficiently high probability (with highest calibration reliability), route to progressively larger models. If the process is exhausted, kick it to a human for further adjudication/checking.

This is a great post and comments here.

I wanted to mention that there has been quite a lot of work in machine learning and statistics over the last few years as to how to analyze large models with non-identifiable parameters, of which LLMs are one such instance. Given that multiple values of the parameters can lead to demonstrably similar outputs (the non-identifiability problem), an important high-level takeaway is that it is typically more productive to shift the unit of analysis from the parameters to examples/exemplars of the input/output.

In my biased opinion, our startup Reexpress AI has already taken this older line of work to its logical conclusion: Non-parametric constraints for large language models that make a direct connection between the observed data and new predictions.

Our website has a link to our macOS software: https://re.express/

(Aside: Separately, we also do some interesting things to make it fast enough for on-device learning. While the full unlabeled training datasets for an LLM are typically too large to process on-device, the standard setup here for a well-specified task is that there is a rather smaller amount of high-quality labeled data from which we derive the uncertainty estimates, and which can effectively be modeled on-device. Thus, perhaps counter-intuitively, deriving such estimates for the largest models available today is often no problem in practice, because we can just estimate against the available labeled data for a given task. Our software makes it easy to do this by ensembling additional models with our on-device models.)