This is a common misconception - "averaging out" errors only works if the errors are pretty rare at any given site. This is true for some types of errors & sequencing technologies, but not universally true. Some types of DNA sequences (most notably homopolymers and other simple repeats) are very difficult to sequence correctly, and X% of the reads there will be incorrect. If X>20% of so, then it may look like real germline variation no matter how many reads are sequenced