News for hackers. Stuff that matters.
Re:Lessons from Optics, the Other Deep Learning by twtw on Monday February 12, 2018 @01:34PM (#16360743)
Another field that might be interesting to compare is analog design. There is a similar stack of theories: lumped element -> transmission line -> maxwell's equations. And yet analog IC design depends heavily on inherited mental models from mentors and modifications of well known topologies. Outsiders think it is black magic. The physics is all understood (nearly) perfectly, and yet knowing the details of QM that explains MOSFET operation helps not at all (or very little) when designing actual useful circuits. The real world considerations of parasitics, coupling, etc. dominate, and extensive formal analysis is not terribly useful. The general methodology is to make changes to the design based on intuition, simple predictive models that give you a direction, and previous experience, and then simulate to see how you did.

A ton of high-quality engineering is done based on intuition, mental models, and patterns learned over years of experience. My hunch is that deep learning will be the same.

EDIT: Just reread, and I want to clarify. I'm not saying that analog design is at the same stage of development as deep learning, or that it is anywhere near as ad hoc. Deep learning probably has a long way to go, but it could potentially end up in a similar state where years of experience is critical and intuition rules.

Re:Lessons from Optics, the Other Deep Learning by wrycoder on Monday February 12, 2018 @03:13PM (#16361634)
A couple of times, I've heard Gerry Sussman at MIT (SICP author) give a talk on how bipolar transistor circuits are actually designed. You don't use SPICE or mesh analysis except in unusual circumstances (e.g. non-linear circuits) or to fine tune a completed design.

As an example, a bias design goes something like this: "Let's see, I'll pin the base at five volts with a resistor divider. The emitter will be 0.6V below that. Then the emitter current will be (5.0 - 0.6) divided by the emitter resistor. The collector current will be essentially the same, so I can pick the collector load resistor to give me an appropriate quiescent point and make sure the output impedance is less than a tenth of the input impedance of the following stage (so I can ignore the latter)."

Re:Lessons from Optics, the Other Deep Learning by analog31 on Monday February 12, 2018 @08:05PM (#16363915)
The classic Art of Electronics by Horowitz and Hill definitely taught this approach. It's considered to be a hard textbook for undergraduates, but is also a joy to read and study for pleasure.
Re:Lessons from Optics, the Other Deep Learning by vvanders on Monday February 12, 2018 @10:49PM (#16364592)
Can't upvote enough, that book is a treasure. Expensive but totally worth it.
Re:Lessons from Optics, the Other Deep Learning by amelius on Tuesday February 13, 2018 @02:46PM (#16370090)
Sorry, but I can't agree. It's basically a cookbook. It doesn't present the material in an analytical style.
Re:Lessons from Optics, the Other Deep Learning by vvanders on Tuesday February 13, 2018 @04:53PM (#16371166)
Is that a bad thing? For me the value wasn't in the cookbook aspect but in the explanation of the practical considerations when it comes to engineering a design.

Knowing a parameter in detail isn't nearly as important as knowing if that parameter matters in the scope of the final design.

The book is also pretty honest about that and goes to pretty good lengths to guide readers to deeper material if it's an area they want to understand in greater depth.

Re:Lessons from Optics, the Other Deep Learning by ehead on Tuesday February 13, 2018 @09:43AM (#16367442)
Seconded. This book should be given to all EE undergraduates, it’s a shame how few people know about it.
Re:Lessons from Optics, the Other Deep Learning by vvanders on Tuesday February 13, 2018 @12:12PM (#16368751)
Yup, I've lent/given away my various copies over the years many times.

A true testament that even in there modern days the principals taught are still solid.

Re:Lessons from Optics, the Other Deep Learning by taeric on Monday February 12, 2018 @03:18PM (#16361672)
https://www.infoq.com/presentations/We-Really-Dont-Know-How-... is the version of the talk I've seen. Highly recommended to everyone.

I have yet to get his classical mechanics book. Gonna have to pull the trigger on that soon.

Re:Lessons from Optics, the Other Deep Learning by wrycoder on Monday February 12, 2018 @10:05PM (#16364426)
I haven't had a chance to ask him why he didn't cover special relativity. Seemed like a great fit for a computation-based course.

Edit: Thanks for that - it's more detailed than the talks I attended (LibrePlanet).

Re:Lessons from Optics, the Other Deep Learning by awful on Tuesday February 13, 2018 @09:14AM (#16367222)
We were taught to do it that way, without having access to spice and computers in general, using the data sheets and maybe a curve tracer if you were lucky.
Re:Lessons from Optics, the Other Deep Learning by joe_the_user on Monday February 12, 2018 @11:49PM (#16364843)
A ton of high-quality engineering is done based on intuition, mental models, and patterns learned over years of experience. My hunch is that deep learning will be the same.

It seems deep learning is in pretty much at that state now (with possible exception of quality). The problem is that this puts inherent limits on what can be done with it. The power of digital computing is the power of modular expansion of objects. Analog circuits and computers don't have that. And current trained deep learning models don't have combinability and modularity either.

A lot of specific engineering subfields involve this "intuition, mental models, and patterns learned over years of experience" but keeping that model of deep learning indefinitely would have to involve a vast proliferation of such subfields each with their limits as the differences in applying deep learning techniques to different subfields become evident.

Re:Lessons from Optics, the Other Deep Learning by rmetzler on Monday February 12, 2018 @03:27PM (#16361760)
Yeah, deep learning needs a design pattern catalog like the Gang of Four book [1] and all which came after.

And while we're on it, is there something similar to analog or digital designs?

[1] https://en.wikipedia.org/wiki/Design_Patterns

Re:Lessons from Optics, the Other Deep Learning by kbob on Monday February 12, 2018 @04:45PM (#16362577)
Jung's IC Op Amp Cookbook comes to mind, though that might be more basic than you want.
Re:Lessons from Optics, the Other Deep Learning by mhneu on Tuesday February 13, 2018 @08:31AM (#16366953)
Question about analog design: how well can projects be transferred amongst engineers? Is the knowledge base relatively similar across individuals, or is it hard for others to modify designs?
Re:Lessons from Optics, the Other Deep Learning by darkmighty on Tuesday February 13, 2018 @10:24AM (#16367776)
Electronics engineering (all engineering as far as possible really) does have a good amount of modularity. So understanding a circuit is about understanding each module, or simply being told their general function. You have a general geometry e.g. a balun or an antenna should follow, but the details very much follow from complex electromagnetic interactions, and I assume are designed through a intuition, simulation and testing loop. So if another engineer needs to slightly modify your design, he will usually just try adding or modifying individual modules to get the desired function -- that could include modifying the geometry of a circuit board analog filter, re-shaping an antenna for different directivity characteristics, etc.