Most people think of Qt as a UI framework but it's also an excellent foundation for CLI/server apps, as an alternative to the C++ standard library and/or Boost.

Qt is split into many sub-libraries so you only link to what you need. For example, one of my projects (pushpin.org) uses Qt but only links to the core and network libs; no GUI/X11/etc.