Does YouTube's system also make a malicious complainer liable for damages? The DMCA does. Here's the relevant section of the law mentioned by krisoft:

(f) Misrepresentations.—Any person who knowingly materially misrepresents under this section— (1) that material or activity is infringing, or (2) that material or activity was removed or disabled by mistake or misidentification, shall be liable for any damages, including costs and attorneys’ fees, incurred by the alleged infringer [...]

https://www.copyright.gov/title17/title17.pdf