The issue is you cannot make a language that uses the feature. It doesn't matter if you are user of Swift or not. The license only grants you the ability to use the patented feature as long as you are using it as part of the licensed software.
The license is irrelevant. Apple doesn't need a license because it owns the patent. Also, the license that Apple grants to others to use its invention doesn't take away any of its own rights to the patented invention.
This is wrong. The license says:
>3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted.
'Each Contributor heareby grants to you, ... licence ... to use ... the Work ....'
However, I am very much not a lawyer. Like Chris, I do talk to IP lawyers as part of my work.