If you use a custom/download able font at all, it would make sense to use one copy (URL) - so every visitor only has to download it once - then use it from cache. This probably also the best reason to use a central CDN over your own - likely some other site already warmed up the client cache for you, with popular fonts (the ultimate in pre-warmed font cache, is of course the fonts the browser/os already have installed - unfortunately the lowest common denominator is a problem here, between Android, ios, os x, windows, Linux).

I'm not sure if Google font service actually does facilitate cross-site caching though - there might be tailored URLs or account?