News for hackers. Stuff that matters.
Re:When Math Gets Impossibly Hard by 3pt14159 on Wednesday September 16, 2020 @03:04PM (#24496205)
I know a military contractor working on some pretty cool stuff, especially back in the day. On one of their projects they had to solve some really gnarly partial differential equation and weren't really getting anywhere. Since what they actually wanted were values (not the algebraic solution) what they ended up doing is building the equivalent circuit and measured what the voltage and current was in the circuit to get the values that they needed for some completely different domain.

I always thought that that way of thinking was pretty neat. Can the same abstraction have a different physical manifestation that's cheap to create? If you kinda squint a bit, it's what we do with computers.

Re:When Math Gets Impossibly Hard by kens on Wednesday September 16, 2020 @06:04PM (#24498529)
I've been doing some restoration on an analog computer. While it seems bizarre now to solve a problem with analog circuits instead of a digital computer, analog computers were a reasonable approach until about the 1970s.

The basic idea of an analog computer is you use op amps to sum voltages. Integration is easy, by adding a capacitor. By hooking up op amps, capacitors, and resistors (usually with a plugboard), you can quickly and easily solve differential equations. (Surprisingly, multiplication by a non-constant value is hard with an analog computer.)

The nice thing about analog computers is they gave the answer almost instantaneously. You could turn the dials and experiment with different parameters interactively. A digital computer in 1970, on the other hand, might take minutes to solve the same equation. This also made analog computers useful for real-time applications such as flight simulators.

A big problem with analog computers is that the accuracy is limited by your components. If you want 0.1% accuracy, you need expensive 0.1% resistors and capacitors. With a digital computer, you can get as many bits of accuracy as you want using cheap components. Basically, Moore's law killed off analog computers.

Re:When Math Gets Impossibly Hard by TrackerFF on Wednesday September 16, 2020 @06:57PM (#24499016)
Just fire up any older electronics textbook, and they'll probably have examples of integrator and summing circuits solving differential equations. At least that's what we had in our control (engineering) books.
Re:When Math Gets Impossibly Hard by sudosysgen on Wednesday September 16, 2020 @09:29PM (#24500196)
In my dad's Calculus II equivalent class in Morocco, one of the question in their exam was to built the circuit to solve a given differential equation. Pretty neat!

That being said, that exam was substantially shorter and easier than the Cal II exam I had to take!

Re:When Math Gets Impossibly Hard by qppo on Thursday September 17, 2020 @12:10AM (#24501107)
> multiplication by a non-constant value is hard with an analog computer

Researching this topic also qualifies you to write technobabble for sci-fi movies. Like "operational trans impedance amplifier."

Re:When Math Gets Impossibly Hard by andreareina on Thursday September 17, 2020 @12:58AM (#24501329)
If you haven't seen it yet, the retroencabultaor video [1] is a masterclass on how to do it.

[1] https://youtu.be/RXJKdh1KZ0w

Re:When Math Gets Impossibly Hard by nullsense on Thursday September 17, 2020 @06:01AM (#24502807)
Re:When Math Gets Impossibly Hard by elcritch on Thursday September 17, 2020 @01:45AM (#24501591)
I actually use a transimpedance amplifier for my work, and it’s even operational most of the time! But one does wonder if techno-babble really does sound like real techno jargon to none experts.
Re:When Math Gets Impossibly Hard by jiggawatts on Wednesday September 16, 2020 @09:20PM (#24500142)
There is practically no scenario where a digital computer couldn't do better if what you wanted was a numerical result and not a symbolic one.

It's simply criminal how underutilised the technique of Automatic Differentiation is! It can solve incredibly complex differentials to machine precision, good numerical stability, and minimal tuning or hand-holding.

Lots of these "difficult" problems aren't, it's just that people aren't aware of the solution because it's not commonly taught in a University setting. Automatic Differentiation is one of those techniques that's "boring" and "numerical", so there are no courses that teach it.

Re:When Math Gets Impossibly Hard by theemathas on Thursday September 17, 2020 @01:36AM (#24501544)
The parent comment is talking about solving differential equations. That's a very different thing from computing differentials. Automatic differentiation doesn't help with differential equations at all.
Re:When Math Gets Impossibly Hard by 3pt14159 on Thursday September 17, 2020 @09:21AM (#24504271)
Yeah, this was a while back. The guy used to work for the South African military during apartheid and later worked on systems in Canada and Asia. I'm sure you're right that digital computers could do the job today, but back then the software and hardware wasn't close to being up to the task.
Re:When Math Gets Impossibly Hard by ampdepolymerase on Wednesday September 16, 2020 @11:28PM (#24500889)
Doesn't literally every optimisation and ML class teaches it nowadays?
Re:When Math Gets Impossibly Hard by jiggawatts on Thursday September 17, 2020 @12:32AM (#24501202)
Actually, no! You would think so, but again, it's a boring numerical technique so it is all too often sidelined in favour of the more "flashy" symbolic techniques.

An awful lot of tertiary education has been caught up in the publish-or-perish mentality. Unfortunately for AD, it is not "impressive". Differentiating hideously difficult expressions symbolically is impressive. Publishing a paper packed full of enormous, scary looking expressions is impressive. Merely saying that "we used AD and it worked just fine" is unimpressive.

For decades now we've been incentivizing intellectual wankery over practical utility.

Re:When Math Gets Impossibly Hard by lambdatronics on Thursday September 17, 2020 @02:01AM (#24501672)
Here's an example from my field: "The derivatives of the finite-build coils are computed -- with almost no additional effort from the authors -- using automatic differentiation." Well, what kind of scientist are you, if you are too lazy to take derivatives? /sarcasm

http://meetings.aps.org/Meeting/DPP20/Session/BP14.32

Re:When Math Gets Impossibly Hard by jackfoxy on Wednesday September 16, 2020 @05:20PM (#24498072)
Isn't that the basis of analog computing? https://en.wikipedia.org/wiki/Analog_computer
Re:When Math Gets Impossibly Hard by lopmotr on Wednesday September 16, 2020 @06:03PM (#24498520)
Engineers used to use the shape of a soap bubble to calculate the torsional stiffness or perhaps stress distribution of a non-circular shaft.

http://www.newsteelconstruction.com/wp/wp-content/uploads/Te...

Re:When Math Gets Impossibly Hard by ghj on Thursday September 17, 2020 @12:34AM (#24501214)
Soap film is also used to solve the steiner problem: https://www.youtube.com/watch?v=dAyDi1aa40E
Re:When Math Gets Impossibly Hard by trenchgun on Wednesday September 16, 2020 @03:19PM (#24496393)
Isn't that just analog computing? Or is it something more fancy?
Re:When Math Gets Impossibly Hard by enriquto on Wednesday September 16, 2020 @03:24PM (#24496482)
This is analog computing, and it is really cool. Yet, I guess if it corresponds to a circuit it should be more an ODE than a PDE (unless your "circuit" involves large pieces of surface of non-uniform resistivity).
Re:When Math Gets Impossibly Hard by 3pt14159 on Wednesday September 16, 2020 @03:34PM (#24496620)
It was a true PDE, not an ODE. The circuit was non-trivial, but though I knew PDEs at the time he told the story, I'd never used them on electronics (only heat / pressure flow, and other physical engineering stuff) so I couldn't really understand what he was saying.
Re:When Math Gets Impossibly Hard by jonah-archive on Wednesday September 16, 2020 @04:33PM (#24497474)
You can definitely do this! Check out Karplus's "Analog Simulation" for examples with conductive fluids and resistance networks: https://archive.org/details/analogsimulation0000karp
Re:When Math Gets Impossibly Hard by marcosdumay on Wednesday September 16, 2020 @03:29PM (#24496548)
Yes. And a numerical calculus software would probably get there faster and with less work.
Re:When Math Gets Impossibly Hard by enriquto on Wednesday September 16, 2020 @03:36PM (#24496655)
Numerical solution of PDE is not really a solved problem. If your PDE involves shockwaves or turbulence (which happen for really simple PDEs) then the methods to compute solutions get incredibly fancy. There's no "black box" solution for numerical PDE, as there is for ordinary equations.
Re:When Math Gets Impossibly Hard by marcosdumay on Wednesday September 16, 2020 @03:52PM (#24496857)
If you can solve with an analog computer, without problems with metastability, oscillations or excessive noise, then you can solve it with numeric calculus too. Those computers have much more strict limitations than digital math.
Re:When Math Gets Impossibly Hard by 3pt14159 on Wednesday September 16, 2020 @04:06PM (#24497073)
I should call the guy up sometime and get the details about what they were working. It wasn't a run of the mill analog computer, but who knows how novel it really was.
Re:When Math Gets Impossibly Hard by klyrs on Wednesday September 16, 2020 @08:48PM (#24499922)
Your description covers pretty much every quantum computer ever conceived. How certain are you?
Re:When Math Gets Impossibly Hard by sudosysgen on Wednesday September 16, 2020 @09:30PM (#24500211)
Not sure you can call any of the actual quantum computers we've made "analog", but if you're talking about the quantum annealment machines, they are generally more limited and take longer than an equivalent supercomputing cluster.
Re:When Math Gets Impossibly Hard by marcosdumay on Wednesday September 16, 2020 @10:27PM (#24500526)
Hum... Quantum computers tend to be digital, not analog. The word "q-bit" is there for a reason, just because the bits are entangled it doesn't make them analog.

Besides, the set of problems you can solve is different from the set of problems you can solve quickly. When discussing analog and digital computers, the speed difference isn't all that relevant, but you seem to be confusing them here. Quantum computers do not solve any problem you can't solve on a classical computer, they just solve some faster.

Re:When Math Gets Impossibly Hard by sudosysgen on Wednesday September 16, 2020 @10:36PM (#24500580)
Well yeah, that's exactly my point. Quantum computers are digital, which is not what the above comment claimed, except for quantum annealers which are closer to analog computers.

What I'm saying is that even quantum annealers aren't really faster at any practical problem than a comparable supercalculator as of yet.

Re:When Math Gets Impossibly Hard by marcosdumay on Thursday September 17, 2020 @09:12AM (#24504176)
Oh, I was intending to reply to the parent of your comment.
Re:When Math Gets Impossibly Hard by petermcneeley on Wednesday September 16, 2020 @04:33PM (#24497457)
Can you provide a simple example of this?
Re:When Math Gets Impossibly Hard by enriquto on Friday September 18, 2020 @05:19AM (#24514662)
The classic example are Navier-Stokes equations, a simplified model of fluid mechanics. Mathematicians do not yet know how to prove that they have a solution, but they can implement discrete versions of these equations that behave somewhat realistically. Yet, it is extremely tricky to get it right; for example, to observe turbulence at the same speed that appears on a real fluid.

A much simpler example where naive numerical methods do not work, is any first-order equation with converging characteristics. For example the inviscid burgers equation u_t + u u_x = 0. Even if you start your evolution with a smooth profile, it will form discontinuities. When you try to discretize it, computing derivatives by finite differences does not work at all, and the simplest upwind schemes have a lot of diffusion (which does not correspond to the model).

Re:When Math Gets Impossibly Hard by btrettel on Friday September 18, 2020 @07:51AM (#24515551)
> Mathematicians do not yet know how to prove that they have a solution

Probably just a typo, but it's not hard to find particular solutions. The question is more whether solutions exist for all reasonable initial conditions. Here's a famous particular solution:

https://en.wikipedia.org/wiki/Taylor%E2%80%93Green_vortex

I've used a variant of this one to test CFD software.

The Millenium prize problem has a succinct statement of what's under question:

> In three space dimensions and time, given an initial velocity field, there exists a vector velocity and a scalar pressure field, which are both smooth and globally defined, that solve the Navier–Stokes equations.

https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existenc...

Keep in mind that this is for incompressible Navier-Stokes with a constant viscosity. For inviscid flows you could get discontinuities as you've indicated with the inviscid Burgers equations, and I know that existence and uniqueness problem for the incompressible Navier-Stokes equations has been proved for certain uncommon viscosity laws (which should be reasonably realistic as a constant viscosity is not quite right).

(And of course there are other complications not considered in this problem like boundary conditions.)

Re:When Math Gets Impossibly Hard by enriquto on Friday September 18, 2020 @08:20AM (#24515741)
> Probably just a typo, but it's not hard to find particular solutions.

Indeed, a fluid eternally at rest is surely a solution :)

Re:When Math Gets Impossibly Hard by creato on Wednesday September 16, 2020 @04:08PM (#24497094)
It depends on what "back in the day" means. Modern computers are absurdly capable compared to what might have been available to these engineers "back in the day".
Re:When Math Gets Impossibly Hard by jhayward on Wednesday September 16, 2020 @03:34PM (#24496627)
and higher precision.
Re:When Math Gets Impossibly Hard by adamnemecek on Wednesday September 16, 2020 @03:41PM (#24496714)
It is. I hope it comes back in the photonic form.
Re:When Math Gets Impossibly Hard by dzolob on Wednesday September 16, 2020 @07:15PM (#24499176)
I knew a guy who built specific surfaces with plaster and threw marbles at them to find some special solutions to nonlinear mechanics problems. He thought of himself as an experimental mathematician.
Re:When Math Gets Impossibly Hard by Loranubi on Wednesday September 16, 2020 @11:32PM (#24500915)
Also see Optical Fourier co-processors for a similar example.
Re:When Math Gets Impossibly Hard by curiousgal on Wednesday September 16, 2020 @04:04PM (#24497035)
Banks do that all the time with monte carlo simulation. Can't get a closed form solution? Simulate every possible outcome and aggregate them.
Re:When Math Gets Impossibly Hard by filmor on Wednesday September 16, 2020 @05:46PM (#24498353)
This is neither how a Monte-Carlo process works (it's sampling the possible outcomes randomly, not simulating all outcomes), nor what is described here (building a physical system where an observable value is described by the differential equation one is trying to solve).
Re:When Math Gets Impossibly Hard by curiousgal on Wednesday September 16, 2020 @06:16PM (#24498639)
In practice the number of simulated scenarii is quite large which is what I meant by "every". Not rigorous though, I concur.

Monte Carlo methods are used to solve partial differential equations, I found both approaches similar in the sense that they look at how the system actually/could behave, instead of looking at the equations solely.

Re:When Math Gets Impossibly Hard by stagehn on Wednesday September 16, 2020 @07:49PM (#24499469)
This is a pedantic response. There isn't an important difference between building a physical system and building a system in code whose generating process is that which you're trying to model. And everyone knew he didn't mean every outcome literally.
Re:When Math Gets Impossibly Hard by seanhunter on Thursday September 17, 2020 @03:36AM (#24502081)
Worked in the securities division of a leading investment bank for 8 years including on the equities exotics and hybrids trading desk. And I can confirm that we used to use MC whenever we had something we couldn't price using a closed form method. That said, it's a lot slower and more compute intensive so when you could use a grid pricer or pde solver or whatever people would use that instead.

You're not really simulating every outcome, just a "large enough" representative sample.

As an aside, my dad was a chemical engineer and Newton-Raphson was his equivalent everything-is-a-nail numerical method. We used to use brent root and monte carlo simulation in the same way.

Numerical methods are awesome in general.

Re:When Math Gets Impossibly Hard by brewmarche on Thursday September 17, 2020 @07:09AM (#24503200)
There is actually even a connection between PDEs and SDEs (stochastic differential equations), namely the Feynman-Kac formula which is used a lot in financial mathematics, but it generally allows you to switch between the expectation operator and solving a PDE.
Re:When Math Gets Impossibly Hard by CPLX on Wednesday September 16, 2020 @06:05PM (#24498534)
You’re talking more about goal-seeking functions right? That’s the one I always see in a finance context.
Re:When Math Gets Impossibly Hard by andrewjl on Wednesday September 16, 2020 @06:48PM (#24498911)
Is this something an FPGA or something similar could do? Would be interesting to know of any home-brew attempts at this out there.
Re:When Math Gets Impossibly Hard by klyrs on Wednesday September 16, 2020 @08:51PM (#24499940)
Re:When Math Gets Impossibly Hard by XMPPwocky on Thursday September 17, 2020 @01:35AM (#24501536)
FPAAs don't really exist as products you can buy, as far as I know (unlike FPGAs, which are a very mature technology to the point that I suspect there's nontrivial (say, >5%) odds the average HN reader owns something with a FPGA in it somewhere.))
Re:When Math Gets Impossibly Hard by andrewjl on Thursday September 17, 2020 @02:33AM (#24501796)
I didn't know about this. It's curious that the article mentions that recently (2016) improved FPAAs were demonstrated. Perhaps an opportunity where there wasn't one before.
Re:When Math Gets Impossibly Hard by bashinator on Wednesday September 16, 2020 @08:02PM (#24499569)
I wonder how difficult it would be to re-purpose a modular analog audio synthesizer for this. Since they're direct descendants of analog computers in the first place.
Re:When Math Gets Impossibly Hard by andrewjl on Thursday September 17, 2020 @02:41AM (#24501831)
There are ASICs out there that simulate analog synths, such Universal Audio. Brings to mind some interesting pure software approaches if that were the case.