From my limited outsider perspective, it seems like Apple's unwillingness to go out of their way to support Linux is what's holding it back. There seem to be a large number of people interested in using Swift on the server, but without first-party support, Swift on Linux will always be playing catch up, as new features will always land on MacOS first and often not be designed with any regard to how they'll work on Linux.