Amusingly, in high school my physics teacher told the class a story about how things had become so complex that no single person could understand everything about the classroom computer.

That was in 1978 and the computer was an Apple II.

It was entirely possible in those days to understand everything from transistors, to digital logic gates, to flip-flops, to clocks, to adders, to registers, to 8 bit computer chips, to assembler, to the primitive BASIC on an Apple II.

Sure. It's just quantum mechanics, classical physics, electrical engineering, and computer science.

QM and classical physics courses (2 years worth) are required of every Caltech student, as they are considered foundational. EE and CS are optional, of course, but not a problem to take.

I admit I struggled with QM.

But how transistors work is very sketchy to me

You can get a hand-wavey understanding quite easily.

Digital circuits use a fairly simple model, and that's not difficult to pick up.

Analog circuits can use a number of more complex models, and they're more of a challenge, depending on what you're trying to do.

Actually designing transistors - defining dopant levels, topology, and process steps - is a very specialised skill that very few people have.

And modern PC motherboards are marvels of electromagnetic engineering as well as electronic engineering. You have GHz signals running reliably on circuit boards with multiple layers. Getting those to work is another very specialised skill.

So it would quite something to find someone who had a professional-level understanding of all the levels: from silicon chemistry and quantum physics at one extreme, to electromagnetic design, to all the different levels of electronic design, to binary machine language and processor architectures, to OS design and application software, to all the various data structures and encodings that make all of the above useful.

You really need a course in Quantum Mechanics to understand it. But a workable description is a couple pages in most electronic textbooks. MIT covers it in their first course in electronics on Youtube.

You need QM to even explain why electrons don't fall into the nucleus. So I'd say if you want to understand anything then you need QM.

There's this quote which I think applies here: if you want to bake an apple pie from scratch, you first need to invent the universe.

True, but transistors do not behave in a newtonian physics manner, whereas we do just fine understanding most physics using conventional newtonian mechanics.

This makes me wonder what Newtonian physics we actually use in computing?

Non-transistor electronics relies on Maxwell's Equations. While not Newtonian mechanics, it is part of classical (pre-Einstein) physics.

QM would be ultimate foundational understanding. A transistor is a valve. Here, problem solved :D

A generic course in QM won't really help you understand electronics any better. Perhaps if you have a background in electronics or if the course makes attempts to link QM with electronics but otherwise it's just relatively abstract physics.

I don't know any program of study that would get sufficiently deep enough to get to semi-conductor quantum tunneling (year 2/3 class) without having some foundational E&M to motivate the QM to begin with (pure math here being a more exceptional but rare case). I think the beauty of QM at that point is that it looks and feels a lot like differential field work of EM and Lagrangian mechanics but maps to quantized numerical spaces, where jumps and little shoots like that happen all the time (spontaneous crystal domain alignment in magnets for example). If you were able to get sufficiently skilled in intro QM, it would absolutely help study in E&M.

If you E&M questions are just node solving, linear algebra and difeq is all you need for E&M to solve "most" work. You do a lot of specialized versions of that in QM and as mentioned above the people doing the actual semiconductor design are so highly specialized that they are often not transferable to other problem domains and require fundamentally different approaches. Not sure any non-PHD study will really help one advance the field in a meaningful way. Study of QM i would say sufficiently enriches all physics verticals that if you are capable, it's worthy of individual pursuit, but also makes total sense to study as part of a curriculum taking place after intro E&M.

Quantum Tunneling in everyday life: https://www.discovermagazine.com/the-sciences/how-quantum-me...

Now that you mention it. After the QM course there was a solid state physics course where quantum effects show up and where it's applied to electronics/semiconductors.

> A generic course in QM won't really help you understand electronics any better.

Caltech had an elective course for freshman about how semiconductors worked up through flip-flops and registers. I took it. It involved QM and yes, tunnelling. So it's doable for mortals.

Very interesting discussion thread re how much knowledge you need or can aquire on complex technology platforms. The goal is key here, I presume.

I read Charles Perzhold's Code, worked through parts of SICP, developed a bit of high level code software, have a very good knowledge of chemistry, lithographic processes (and QM, half forgotten by now). And yet it is not enough to get a good understanding of the modern computer - EE, compilers, OS de, networks, ... wiring the while thing together, et cetera.

I see the point in historical approaches, but it is quite time consuming. At the end, you will most likely attack problems at one to three levels in the "stack".

What would you consider to be an above average conceptual tool and skill set for the modern hacker?

Yes, I can see how it would help there. I was more thinking of the situation where you take a QM course and don't apply it to an electronics problem. I agree that it's doable for mere mortals, but it might take a fair bit of time!

Do FETs really require QM to understand them?

The handwavy explanations you'll find in electronics textbooks are sufficient to enable you to effectively design electronic circuits around them. But personally, I found those explanations inadequate.

Kind of blew my mind when I learned they are analog... I previously thought of them as one of the most basic digital components, because they're used in every digital circuit.

My job for the past 30 years has been designing ASICs, so all the analog stuff is not my concern. Prior to that I designed at the TTL level. I'd spend months working on some project, thinking in terms of logic an timing, and eventually we'd have boards made. We'd stuff a few prototypes and set them up on the lab bench to commence debugging.

Every time at this phase I'd suffer the existential horror the first time put an oscilloscope to the circuit -- the clock wasn't square! Signals would ring, and other signals which hadn't transitioned would sometimes bounce a bit. Setup and hold times could be pretty well estimated, but it was still an estimate. Then there were phantoms that might appear on the scope or real issues that would not show up on the scope because the probe lead was enough to make what was measured not the same as what was actually going on.

That reminds me of working on the stabilizer trim system on the 757. I got used to thinking of the parts as rubber, bending and twisting on the high loads an airplane has in flight. So much so that when the hardware was finally available, I was surprised at how rigid and immovable it felt under my hands with my puny human strength.

Hi,

Your work sounds really interesing to me.

I am wondering about the reasons for these issues that you mentioned. Could you please mention them here ? It might serve as a nice pointer to do better hardware design debugging I believe(I have close to zero experience in this. Hence the interest).

Also, are there any formal systems that verify your TTL-level designs ?

Thank you.

Sure. There are many reasons why a real circuit falls short of the idealized abstraction.

A truly squarewave clock, or any rising/falling edge that happens in zero time would have infinite bandwidth, and no real circuit can have infinite bandwidth because of capacitance, inductance, and resistance. Even though the power rails of a TTL chip are 0V and 5V, when a fast transition happens, the signal can experience reflections because the impedance of the driver and the copper trace are different, and because the end of the copper trace isn't terminated with impedance which matches the trace. As a result, the edge takes finite time to transition from 0V to 5V, but the signal can actually exceed 5V because of a reflection. Likewise, falling signals can go below 0V.

Also, the signal at one end of a wire will not be the same as the signal at the other end of the wire. It isn't even just a simple time delayed version. Eventually they'll come to equilibrium if the signal is stable long enough.

You can read about transmission lines here: https://en.wikipedia.org/wiki/Transmission_line

OK, another defect. Imagine a TTL part with 8 buffers in it, say a 74ACT244. Say all the outputs are low, and at some moment seven of those inputs switch from low to high. The corresponding outputs must drive enough current to switch their loads from low to high as well. Each of those outputs draw current to charge their load from 0V to 5V, and all of the current flows through a single power pin and a single ground pin. Because of of non-zero resistance and inductance, the voltage seen by the transistors on the chip is shifted slightly from the voltage seen on the PCB. That can cause the output which isn't changing to sink/source current (depending on the direction of the change) causing its output to bounce a bit. Imagine having four people on a trampoline, with one standing still and three jumping at the same time. That "standing still" person will still bounce up and down because of the others causing their common ground reference to shift.

Another defect. A chip's spec sheet lists a bunch of requirements for proper operation and guarantees of the chip's behavior if the requirements are met. Say a flip flip says it has a 5ns setup time, that is, the data in to the flop must be held stable for 5ns before the rising edge of the clock arrives. If this requirement isn't met, the chip's behavior is unspecified (it might capture it, it might not, it might capture it but it might take an arbitrarily long time to appear on the output, or the output could even oscillate). During the design phase one adds up the worst case propagation times for all paths from point A to point B and makes sure there is at least 5ns to spare to guarantee the requirements. But in real life, a clock trace runs all over the board and has many loads, each presenting some amount of capacitance. Kind of like relativity, different chips will see the rising edge at different times. Looking at a given chip, the clock edge and data inputs are each transitioning from high to low or low to high over the course of a nanosecond (and often more). At what point on that slope is the signal really a "0" or a "1"? Sometimes those signals are clean transitions because of reflections, it might go from 0.7V before the clock edge, to -0.5V after the clock edge, to 2.2V (neither logical 0 nor 1), stay like that for a few ns, then move to 4.2V for a few ns, then 4.0V for a few ns, then 4.6V.

The most maddening is the fact you can't always trust what the oscilloscope tells you. The oscilloscope has finite bandwidth; 100MHz scopes were common in the 80s and early 90s when I was using them. The 200MHz scopes were a lot more expensive. You might think, hey my clock is only 25 MHz, 100 MHz is plenty ... but a fast signal edge contains harmonics that are a multiple of that 25 MHz base frequency, so the scope tends to act as a lowpass filter. The scope probe is typically more than a meter long, and there is a clip on the probe that must be attached to your circuit ground. That is a 2m loop of wire, and in the best case it would take a signal a few ns to travel the length of that wire, and any stray EM fields can induce signals on that probe that aren't in the circuit (though obviously the probe designers take pains to minimize such effects). When debugging and a signal looked unexpectedly weird, a common tactic to avoid sinking too much time chasing phantoms was to change the probe, change the channel, or simply waggle the lead wire around to test the sensitivity of the displayed waveform on stray capacitance and such.

Testbenches always had a few cans of refrigerant on hand. If you suspected a chip might be flaky, probe its output then spray it and see if the output is affected by the chip temperature.

The list goes on...

Your explanation is quite amazing. In my undergrad, we had learned about Transistors, TTL, flip-flops etc., but as individual building blocks. Your reply paints a very good picture about what can go wrong they play together. Is there a book that you would recommend to learn more about this ?

Also, Honestly, I thought reflections would be negligible when we are dealing with mA and 0-5V ranges. But your reply makes me think that there is more to it

Finally, thank you for your reply. I gonna come back to this answer in the future when I have a lab setup that is more than a solder and some screwdrivers :)

Oh, I forgot to mention that when you probe a circuit, the presence of the probe can affect the operation of the circuit.

On RF designs it was not uncommon to find seemingly needless small valued components at certain probe points. The idea was these modeled the reactive load of a probe. When you needed to probe it you'd remove the components and attach your probe and the circuit see approximately the same load so you could have more faith that what you were seeing more closely matched what was happening when you weren't looking.

Oh gosh thanks!

I have seen such small valued components, looked their values up and thought, "this is insane".

(I wish I could say I thought, "I must be dumb, someone must be much smarter than I am", but if I hadn't embodied a certain kind of hubris I would not work in software.)

A nice video showing some real world effects of non-ideal probes:

https://www.youtube.com/watch?v=zodpCuxwn_o

Way over my head, but damn it sounds interesting :D

I don't know that there is anything truly digital in the universe, except perhaps quantized components like spin. Digital computers are an abstraction implemented using error control and a number of other things, but they ultimately fail like analog objects.

Well yes, as a kid at the time I remember thinking I understood pretty much all of it except the switched mode PSU and Woz's wacky floppy controller.

What about hi-res graphics? Apple II hi-res graphics were especially wacky.

Woz could understand it. After all, he invented it. He wrote the code for the Apple II. I bet he understood every aspect of it.

Nowadays you'd have to know how a modern lithography machine works. There's a reason why ASML is a highly valuable company. You can perhaps understand the basic ideas, but to implement them is another matter entirely.

That's a surprising example. From reading his autobiography and other sources, it's my impression that by virtue of having designed it from the transistors to the software, Steve Wozniak did in fact grok it fully.

Wozniak has not designed it on the transistors level, he used MOS Technology 6502 microprocessor. Though I agree he could probably understand it on the transistor level if he wanted/needed to.

Ok, so maybe one single person...

You misunderstood my post. The point was my teacher was wrong then and possibly similar assertions now could be wrong too.

Ah, that went entirely over my head then.

What we can understand is partially a product of how good we've become at teaching it.

Except multiple people understand everything about the Apple II :)

>So the improvement of Moore's law (RIP) ends up going to programmers' salaries instead of to performance improvements.

This statement doesn't really make sense. If programmers spent the time to optimize everything instead of gluing a bunch of inefficient libraries together, software would cost even more than it does now. So that's even higher salaries because there is even more business competition for the same pool of available programmer hours in the workforce (and that doesn't even take into account the fact that you would wipe out 90% of the programming workforce if writing assembly is required).

Moore's law means that you can afford inefficient code, so it's taking money from programmers and keeping it in the businesses that would have originally had to pay for a hand-tuned piece of code.

Moore's law for performance has been dead for 15 years. No one's noticed though.

There is no "Moore's law for performance". Moore's law is a statement about the number of transistors on a chip, nothing more (Moore?). And sure enough, the Threadripper machine I built this year has about double the number of transistors as the Threadripper machine I built last year, which has about double the number of transistors to the one before that... (although this is partly a function of increasing budget; Moore's law specifies a two year doubling).

But sure, if you want to talk performance - the fastest CPU this year, the Threadripper 3990X, is over 100x faster than the fastest CPU of 2005, the Athlon 64 FX-57. That's about 7-8 doublings, or pretty much spot on.

Faster how, exactly? The clock rate is ~4/3. It has 128 half-cores, but the benefits will depend on the workload. :-)

For a very long time between 1970 and 2005 doubling the transistors halved the time taken for any task. We're well past this point.

If you don't think so try and calculate a_{i+1} = rand_int() a_i mod 232 on a machine from today and one from 2005. Then do the same with a machine from 2005 and 1990.

My CS systems PhD advisor used to say, 'you waste the abundant resource' in systems design. Lots of CPU cycles available, lots of bandwidth available (except when not!) - so waste it and save programmer time.

At least, in theory.

It's not about knowing assembly or not. It's resources, time, cost, money, ROI.

In my country, water consumption was not measured prior to 1989. Households just paid a lump sum and the price was kept on 1953 levels.

Yes, once measurements were introduced after the Velvet Revolution, water consumption by households fell to half almost overnight.

Even after knowing assembly, there's probably significant optimizations that can be done if you design chips and ISAs that are specific to a certain problem. There's "waste" at every stage of abstraction.

That's not waste, that's a tradeoff.

It's a choice you need to make when developing.

I had to build a one-time export yesterday. Am I gonna spend a lot of time optimizing the speed of it? Or am I going to make it work and get the export to the client faster?

It would've wasted my time and the clients money if I optimized the queries. Sure, the export itself took longer. But overall, the task was done quicker.

If the client wanted this export to run on their server frequently, I would've spend the time making sure the export itself runs faster.

It's especially relevant when thinking not at the level of computer systems but business systems - since for many tasks you anyway have a combined computer+people process that works for some goal, and there's a tradeoff between computers and people for choosing the level of automation you want.

For exmaple, in your example of a one-time export it might also make sense to write code that does not meet all requirements and explicitly can not handle certain edge cases, if it's more effective to fix these cases manually than try to automate that.

Exactly. Yout goal isn’t to optimise the code, it’s to optimise the overall cost/benefit of whatever you’re doing.

Reminds me of Jevons paradox:

> when technological progress or government policy increases the efficiency with which a resource is used (reducing the amount necessary for any one use), but the rate of consumption of that resource rises due to increasing demand

https://en.wikipedia.org/wiki/Jevons_paradox

I think the problem is actually the opposite. There aren't enough abstractions that help programmers write good code. So when a programmer has to do something complex, they end up doing the easy thing which is just importing every library they need. If instead, there was a way to say "only import this when I'm doing this action" and there's no way for the programmer to explicitly say what to import, then that problem almost disappears.

As a programmer I’m mostly measured on how fast and not buggy (in that order) I can ship. No one is profiling my code or asking O(wtf) it takes.

So from crap import * is pragmatic and practical. Short term at least.

> No one is profiling my code or asking O(wtf) it takes.

That you know of.

But the compiler is going to shake the tree and not include anything not being used right?

Not sure if this is sarcastic, but...

1) In dynamic languages, it's not possible to detect whether a function is used or not in the general case. For example, consider string accesses on objects. If the compiler is not sophisticated enough to resolve the set of possible strings at compile time (or such analysis would unacceptably increase compile times), then you can't shake out unused methods on that object. [1]

2) For languages like C and C++, the compiler cannot tree shake because it only knows about a single file at a time (translation unit, to be precise). You would have to rely on link-time optimizations to effectively tree shake, but LTO is not well-supported by all toolchains.

Tree shaking also has a cost that I mentioned earlier -- it increases compile times. Both LTO and tree shaking in dynamic languages increase compile times superlinearly [citation needed] wrt. the size of your application. As other commenters have mentioned, it's better to avoid including unnecessary libraries in the first place.

There are no cost free abstractions, and that applies to tree shaking as well. https://www.youtube.com/watch?v=rHIkrotSwcc

[1] For the pedants: yes, I know resolving the possible set of values (stricter than "all possible values for that type") for a variable is undecidable in the general case.

You could perhaps better use a lazy-loading strategy instead of a static analysis. (But this would change the semantics in case of an existing language that allows side-effects while loading modules, unless you have a lazy strategy for them too ...; and then there are the errors you'd have to deal with)

To truly achieve the same thing as "tree shaking," the function call overhead would be abysmal. You'd have to check whether the module was loaded already (with synchronization if your program is multithreaded). For single threaded programs, you could avoid this by hot-patching your machine code, but there's no way around some synchronization barrier in multithreaded programs [1]. In JavaScript (or any language where you want to avoid sending a large bundle over the network), you'd incur the latency of a network request for the first call to any function.

You're right that people are already splitting apps into bundles, but that is usually done at the page level.

[1] You could probably avoid having to take a mutex after the first call to the function with self patching code, but that sounds incredibly ugly and could have other implications (self-modifying code could be detected as a virus; could be used as a gadget to exploit some other vulnerability).

Isn't self-modifying code already the norm, with JIT compilers?

I am under the impression that JIT compilers do not modify the compiler’s own bytecode. They write generated code into a separate data region and mark that region as executable. If the code needs to modified, then control transfers back to the compiler which can mark the region as writable again.

The same can be done for a binary and it’s own code, but I wonder if it’s used as a signal in antivirus protections if done too often.

https://en.m.wikipedia.org/wiki/W%5EX

... which is why JIT compiled code is slower in some circumstances.

The ol' sufficiently smart compiler myth. It's a classic.

See the Elm to JS compiler, it does deep unused code elimination at the function level so you only ship the code you actually use (or the functions u actually use, even inside libraries).

The closure js compiler is also pretty good if you prefer writing js.

If we only start using better languages in our applications we could improve performance quite a bit.

Dead code elimination has been available in C compilers since... so long ago I can't remember. It's not that compilers can't do it, but much like the halting problem, they really can't catch it all.

I can’t tell if this is sarcastic or not. Tree shaking isn’t a silver bullet and really only works under specific circumstances. Better to not include the library to begin with.

Just program in Fortran IV then the linker will only include routines that are called.

> There aren't enough good abstractions

> I tend to think that if more people understood how to program at this level, the weight of software and websites would be a lot less.

I’ve done assembler work, I’m perfectly competent at it. I would never try to build a website in anything low level like that. It’s optimizing for completely the wrong set of problems. The world has lots of network bandwidth, cpu cycles and volatile memory. You won’t find many situations where having a better performing website is worth sacrificing the savings in human labor you get from high level abstractions.

I have a degree in electronic systems engineering. At college I learned about both digital and analog circuit design and fabrication. Had my fair share of FPGA and Assembly programming. Built some microcontrolled printed circuit boards. In summary, I believe I have a reasonable understanding of how computers work underneath, and I'm not interested in using anything other than a high level programming language for consumer application development as well ;)

The big reason that everything is so slow is that we just can't standardize on things. For example assembler: There are many to choose from! So we solve this by building slow abstraction layers.

But we can't standardize because technology is improving so quickly. Any standard would be quickly obsolete.

If we ever standardize on one CPU, one screen size, one GPU, etc etc. Then we can make big gains by hyperfocusing on this.

> So the improvement of Moore's law (RIP) ends up going to programmers' salaries

If businesses demanded that everything be of optimal performance, developer salaries would most likely increase because that would increase demand for developer labour in the market without increasing its supply. And I know that many developers would be more than happy to do this (performance optimisation being one of these classic 'nerd sniping' tasks that really appeals to many programmers) but are simply not granted the time to.

And there is a reason that hardly anyone's boss is telling them to eek out every bit and clock cycle: they can't afford it, or at least don't want to pay to do that. So the "benefit" of doing this actually lands squarely on the business, if you want anywhere to look for wasted resources.

As someone who used to work in banking software I can tell you where the 1MB of crap comes from - many banks create their web apps out of ready to use components, which are designed to be completely independent and this brings a huge weight penalty.

Oh spare me. If you went out to write a secure banking website in assembly code you'd come back in 20 years with something even more bloated than we have now. Why? Because the first thing you'd write is a higher-level programming language suited to actually building websites. Then you'd write a web framework on top of that web. Then you'd write a bunch of buggy, insecure crypto code. Et cetera et cetera until you've reinvented a much worse version of the stack we have now.

I'm not convinced. Doug Engelbart's lab did essentially that with a dozen or so engineers within 2-3 years before the famous 1968 demo. I'm not saying we should do that for a banking website, but IMO when it comes to Frontend development there is an opportunity to rethink and start over, maybe on top of wasm.

I have known from the transistor level through to the mouse moving on the screen, just not all at once. If I put the effort in then I know I could have a solid grasp of all stages at once but it would be a waste. If you're programming in assembly then you're dealing with instruction sets. You don't need to know how transistors make up a NAND gate to program with instructions, or even how gates fit together to make higher order components. You just need to need you have a processor and memory.

To write highest performance assembly code you need to understand more than the instruction set. You need to understand about cache hierarchy and how cache lines are filled, about microcode and how to fill the pipeline, including how to exploit hyperthreads, what the consequence of different workloads is on boost speeds, how the vectorized instructions map to hardware and when the trade-off is worth it, etc... Not to mention that you really also need to be a GPGPU expert so you can trade off work to the graphics hardware when necessary.

One can write assembly while knowing only a fraction of that stuff, but the compiler will probably outperform that kind of assembly, so there is little point aside from the fun of it.

There is a new upcoming book by Matthew Justice to show and teach people of how computer really works:

https://www.penguinrandomhouse.com/books/645954/how-computer...

The book is still in writing stage thus not sure about the quality though but by looking at the table of contents it's looking very promising.