You'll want an Arduino that has headers and a usb connector already soldered on and the usb cable to fit it. That's really the bare minimum. From the article, this one looks like it would fit: https://www.sparkfun.com/products/11575 and only $9 on sale day.

Now if you want it to do something you can see other than in the logs from within the IDE, that's when you start needing more stuff. Generally a breadboard, wires, and some components like leds and resistors come next; for making blinking lights, who doesn't like those?

As far as the computer, it can be best answered here but is just about anything you have: http://arduino.cc/en/Guide/HomePage in the getting started section.

Pretty much. I set one up for the first time a couple weeks ago and it could not have been easier. Bare minimum "hello world" would basically just be a board, a USB cable, a computer with a USB port, and an LED.

As FYI (you may already know this @morganherlocker, so this is for the n00bs) there is a controllable LED built into the UNO (and similar boards) attached to pin 13, so this is possible without any additional parts. In addition to LEDs and a breadboard, some push buttons, and maybe a photoresitor or a rotary potentiometer to play around with analog inputs are all good starting items.

If you get a normal Arduino (there's a bajillion variants, so I won't address all of those) you need any computer made in the past 15 years running Linux / Windows / Mac, and a usb -> usb A connector. Most Arduinos include that. I suggest, however, you buy some electronic parts - a couple of LEDs or sensors - via a kit and a jumper kit (a bunch of pre-bent color coded sized wires) and a breadboard to experiment.

Also, check out for local hackerspacers, as there will definitely be people that to set you on the right path.

I like the BoArduino USB[1] for breadboarding. Use something like that if you like breadboards and don't intend to make use of any of the shields.

[1] https://www.adafruit.com/products/91

You could try codebender ( https://codebender.cc/ ) which is a browser based IDE that requires a simple browser plugin to be installed to communicate with your arduino.

Linux, Windows, and Mac are all supported by the tools. Additionally the tools are very light weight, just about everything can run them no problem.