You can load it conditionally for mobile safari only. IE, something like this (using the lightweight head.js loader):

        // load fast click if mobile safari
        if(navigator.userAgent.match(/(iPhone|iPod|iPad)/i) {
          head.js("js/fastclick.min.js", function() {});
        }

or include it in your main compressed js assets but only execute if mobile safari