The (old) C# compiler, as the VB compiler, was actually part of the .NET framework starting with .NET 2. Since .NET comes pre-installed on Windows, you also have a compiler. However, the new C# 6 compiler is no longer included in the framework, I think. When you install .NET 4.6 it even adds a notice to the old compiler's greeting message:

This compiler is provided as part of the Microsoft (R) .NET Framework, but only supports language versions up to C# 5, which is no longer the latest version. For compilers that support newer versions of the C# programming language, see http://go.microsoft.com/fwlink/?LinkID=533240.

Yes, that's right.

It is? I had to install Visual Studio manually the other day in order to compile C#. Or are you talking about `csc.exe'?

I did mean csc.exe. On my machine csc.exe came pre installed but was not on the path.