If you go the digital route, most (software) hackers can be creating fairly elaborate systems in an afternoon. They do the bulk of the work in the software, rather than futzing with resistors and capacitors.
Imagine the difference between creating a LED array to mimic the KITT car's LED light scanner in analog and in digital. Analog would be a nightmare, digital would be a great afternoon of fun on very limited hardware.
Yes, eventually the person will run into the limitations of not understanding the analog well enough, but since they weren't bogged down in details, they may be excited enough to keep with the hobby.
"What's a good book to teach me the basics of electronics, e.g., why do circuits need resistors, when do I use a capacitor, etc?"
I.e. he wants to learn how analog electronics work, amongst other things.
I own The Art of Electronics, but find it a bit theoretical. If you just starting out you might even wants something aimed at high-school kids (no offense intended)
If you like to learn by being taught (vs. self-taught) then sign up for class at a local school, or if you don't have the time/money then try sneaking into some lectures at your local university (I don't know how illegal / difficult that is in the US, so beware). If you do want to be self-taught then see if you can borrow the lecture notes from someone who has taken electronic engineering. Or try this http://www.varsitynotes.com/electrical_engineering/electroni...
I would highly recommend buying a kit (lots have been mentioned) and if necessary supplementing it with components from your local electronic components store. And take things apart,
I'd also recommend building something that has an interest to you outside of electronics, e.g. if you're into cars then build a radio controlled car.
Some people prefer the details first, some prefer to staart easy, make a bunch of mistakes and learn more and more details as one goes along. I'm one of those people, and that's why my advise fits that mould.
The advantage of starting with VB is that it's very easy. However, these days there are other languages that are easy to learn and that are more powerful than VB, i.e., Python.
I think that the best way to learn electronics is to start with the very basics (resistors, capacitors, coils, voltage dividers, simple RC and RL circuits, etc), then learn something about op-amps, then move on to nonlinear devices such as diodes and transistors. No math is needed, and simple circuits are easy to build. Building an analog amplifier is not hard, and it's quite rewarding to design and build something that actually works. If one is really devoted to the art of electronics, building a tube amplifier is even more rewarding ;-)
After the basics of analog electronics, one can move to digital circuits, learn about interfacing, what the difference between RTL and TTL is, etc, etc. Playing with logic gates on a breadboard is fun and instructive. The basics of DAC's and ADC's are fundamental too.
Finally, one is ready to move to microprocessors and stuff like the Arduino. Knowing the basics, one can actually build cool stuff with microprocessors, simple actuators and sensors. One knows how to build and connect sensors to microcontrollers, one knows what's happening in the ADC, one understands what makes it all work.
I believe that starting with the Arduino board right away is like choosing Visual Basic as a first programming language: it hides the interesting details, and it prevents one from seeing the big picture. Just a personal opinion, of course.