Linux (and probably also the BSDs) goes beyond "case-sensitive": filenames are a sequence of bytes, giving special meaning only to the NUL byte (as the terminator) and the forward slash byte (as the path separator), and to the "", ".", and ".." filenames. From an implementation point of view, it's much simpler, and makes it easier to have high-performance code.

Case insensitivity adds a whole set of complications. The operating system has to either canonicalize all filenames, or preserve them (case preserving) but have an equivalence class of filenames. Canonicalization can be problematic (like ß -> SS, or perhaps ß -> ẞ? And let's not think of the dotless i), and to make things worse, can change with newer versions of the Unicode standard (as new characters and their case mappings are added); at least NTFS works around that by storing a case mapping table in the filesystem metadata, so a filesystem created by a newer Windows release can have a different case mapping (and you can have different case mappings for each filesystem in the same machine). It can also confuse software: even with a case preserving system, a program can open a file named FOO for writing, and not find it later when listing the directory (because it already existed as lowercase "foo"). This kind of thing has led to security issues in the past.