You lose the license granted in the PATENTS file, but you might not lose the license granted in the LICENSE file. (IANAL, so no idea whether that's true.)