A Wenger or Victorinox base model Swiss Army knife is the classic cheap and useful, keep-it-with-you knife.

If you want a dedicated knife with a bigger blade, a Buck folder with a plastic handle will cost about $30 and last a long time with minor care.

Next step up from there would probably be the least expensive Spydercos, or a multitool if you want more stuff.