Chris has posted the relevant papers in a sibling comment, but let me try to give an intuitive explanation. Say you have some sort of electronic device like an amplifier. It is made out of many individual transistors and parasitics, but its time evolution is overall guided mostly by the amplifier characteristics its supposed to implement plus boundary effects and corrections from the parasitics. So what you try to do is to learn a simpler system of equations that captures the time evolution plus a (highly-nonlinear, complicated) projection that recovers the signals of interest (if you don't care about reproducing interior signals, you can usually get away with a simpler model). Of course it's not obvious that this should actually work (though of course picking a model of the same size as the system plus an identity projection is always possible, so it'll always work for some size of the model being learned, just at that point you don't get a speedup), but our initial results show that it does seem to be very promising.

Since this is based on ML, I would have trouble trusting the results. Can you verify the results with standard mathematical methods? E.g. solve a linear system using AI, then compute the residue and meaningfully interpret it? Would it be possible to apply ML techniques iteratively, and let the error approach zero?

Sure, you can always just run the baseline simulation and compare error rates. You can also do fancier analyses to get some rough idea of robustness and maximum error rates (over your particular parameter domain).

Would this integrate with e.g. Measurements.jl?

The error propagation won't be through measurements, because that blows up linearly. There's other approaches which give much more promising results.

From an implementation perspective of course, so you can get error propagation on the baseline simulator. For the actual ML part, you'd get Measurements complaining that it doesn't know how to propagate bounds through the surrogate. You can do some estimation of what that propagation would look like, but it's a research project.

It seems that there are two related but distinct ways to simplifying a model:

1. projections / disregarding some dimension, followed by some model in this latent space, followed by an injection back into the native space. The model is simpler by virtue of operating in a lower dimensional space.

2. using a "reduced-order model" or "surrogate model", but still operating in the same input/output space. The model is simpler by virtue of, e.g. using fewer derivatives or delays.

And with 2, you can have have the simple model e.g. arise from first principles, and learn a _residual_ correction on top of it.

And it sounds like the comment contains a bit of both. Am I getting that right?

A bit of both.

Do you think such an approach could extend to the final verification stage of a large post-extraction chip? My impression is that speedups are most sorely needed in this final sign off phase where the number of nodes explodes with parasitic R's and C's, especially in modern technologies. Simulation times in weeks seem necessary now for sufficient verification accuracy of analog/mixed signal chips.

Yes, we're looking at it. One of our benchmark cases for this work, will likely be a post-extraction structure from one of the Sky130 chips (not that it wouldn't work in a more moderns process, but the NDA issues become complicated while this is still a research project).

There are ways to surrogatize portions of a simulation, replacing large portions with ML-trained surrogates. There are already demonstrations in different domains (on highly stiff differential-algebraic equations) showing that you can get these kinds of speedups at around 99% accuracy.

https://arxiv.org/abs/2010.04004

https://arxiv.org/abs/2103.05244

> can get these kinds of speedups at around 1% accuracy

Shouldn't this be at around "at around 99% accuracy" or "within around 1% accuracy loss"?

Thanks, fixed.

the TLDR is that for a lot of high dimensional PDE models, you get a better trade off of speed vs accuracy by using a NN for part of the model that is inefficient to calculate the full physics for. This is already having a lot of success in climate modeling, where NN based solutions do a better job of dealing with some atmospheric effects than previous efforts.

Sounds similar to what stockfish (leading chess computer engine) does. In newer versions, position evaluation comes at least in part from a NN, which lead to an improvement in playing strength.

edit: similar in approach, not as a 1:1 mapping. Replace a deterministic model with a faster, slightly fuzzy one.

This can be a useful connection, but I believe it's worth fleshing out in case it is misleading

What's the alternative to using a NN for position evaluation? I can think of two:

1. Do minimax search until you have a winner or a stalemate. Then you have the exact value of the position. Well, this is the problem we're actually trying to solve to begin with, and it's also impractical to do for chess and any interesting game. This is what necessitates an approximation to position evaluation.

2. A human expert writes a position evaluation function. It determines a huge handful of features, a simple example being how many pieces I have - how many pieces you have, and some way to combine those features into a score.

In surrogate modeling, you can get ground-truth data to evaluate your approximation against. You're approximating another model that you can compute, it's just too slow for practical use.

In chess, we don't know THE position evaluation function. We can certainly get data about it, but we don't know it in the same way we know PDE models.

To be clear, I am not saying we _know_ the PDE models are accurate with respect to reality. That's the science part, to determine if the model arising from empirical evidence or first-principles that are themselves arising from empirical evidence, actually summarizes empirical evidence.

Chess position evaluation is always going to be somewhat subjective and statistical. Electronics simulation is neither; it's physics.