Petzold's book "Code: The Hidden Language of Computer Hardware and Software" is good. It starts too gently, and ramps up too quickly.

(http://www.charlespetzold.com/code/)

The Art of Electronics Lab Book has plans for building a computer and it's pretty well documented, although very old.

(http://www.amazon.co.uk/Laboratory-Manual-Electronics-Paul-H...)

Another good way to learn is to find a circuit diagram and run it by pencil. Just like people learn programming by writing code on paper and running the software with a pencil; writing the variables and such. Here's one (http://www.nathandumont.com/sites/nathandumont.com/files/ima...) (not a great example) but you try to work out what values you need where to get various bits working.

When you get a better circuit diagram you can create a memory map. Here's a link that describes what I mean. (http://www.clear.rice.edu/elec201/Book/hardware.html)

Good Luck!

"Elements of Computing Systems" is relatively compact and freely available online:

http://www1.idc.ac.il/tecs/plan.html

You might learn the most by doing the first 6-8 projects.

Computer Systems: A Programmer's Perspective by Bryant and O'Hallaron might be a good resource. It also covers things like assembly code and caching that you might not be interested in, but its a solid introduction to the low level details on how computers work. Its used in my university (and many others) in the Introduction to Computer Systems course.