That's what we did when developing our original Embedded Linux stuff. Once you get it working it's really nice. You have your standard tools, version control, etc running locally on your system.