It's certainly a field plagued by some intentional obfuscation and annoying jargon (krigging anyone?), but there's real value in it like anything else. You can prove that to yourself easily. Choose a noise process and generate x,y samples off a line + noise. try ordinary lst-squares and then some gaussian processes...

Basically the value of statistics lies in the fact that there's some human-tractable set of noise processes and correlations which appear universally in a wide spectrum of real phenomena.

Knowing the real mechanism which generates the noise is always better, but not always tractable.