If you need something more selective, BetterTouchTool (which you might already have) has a sleep trigger that allows you to disconnect specific devices.