At age 13 I moved from San Diego CA to Brooklyn Park MN. Shortly after the move I spent a summer helping my stepfather's grandfather build an addition on his house. He lived in Park Rapids MN which in a very isolated, rural area 4 hours north of Minneapolis. After the first few weeks there I started to feel like I was going crazy. I had no history of mental illness but I was suddenly feeling increasingly anxious and agitated with each new day...then I put a hat on.
After I put on the hat, I felt normal again. I had always lived in cities and never had an unobstructed view for more than a mile or so. Out in the country I could see for many miles and it had an effect on me. The bill of the hat blocked my view.
Or it is there (and maybe everywhere) but we can't hear it
One thing that's noticeable when it's really quiet (and having lived in a place where it is silent at night) is that the threshold of hearing changes a lot. Something that's barely audible in normal conditions can "scream" at night
Example: the sound of soda fizzing in your mouth.
On more than one occasion standing outside the house on the mesa outside of town, the utter silence of the place got to me enough that I had to go back inside and talk to somebody. Perhaps my mind would have eventually filled the gap with a hum.