The kernel developers aren't on the hook at all here. No one told you to buy a specific device.
Except for devices that are hackable in practice, but aren't much so because of this heartwarming attitude.
It's not unusual that someone had managed to build the kernel that boots (and even works well) but doesn't know how to interface some hardware. E.g. a hardware NAT engine of a SOHO router, so FLOSS firmwares have to NAT in software. Or some sensor on a phone. The secret is in the kernel and that kernel's source's supposed to be available but it isn't.
Your beef is with device vendors, not Linus.
Sure thing, if I don't have access to the kernel, then it's all irrelevant. We only consider the cases when the device is actually hackable (despite whatever its vendor says).
I (the end-user) get a device with a blob that looks like a Linux (the kernel). I don't get any freedoms (but to whine to vendor). And I get a message from the very head of kernel developers that they aren't interested in me having anything.
Now, it's also true most users don't give a damn about their freedoms. And those who do are such a tiny minority they can be just ignored - unlike the valuable corporate partners.