The issue here isn’t that youtube-dl infringes copyright, it’s that (according to the RIAA claim) its purpose is to “circumvent the technological protection measures used by authorized streaming services.” Even if youtube-dl was tested exclusively on public domain videos, so that the developers never infringe copyright, it could violate 17 U.S.C. 1201 [1] if the RIAA succeeds in proving that it was “primarily designed” or “marketed” for circumventing technological protection measures.

[1] https://www.law.cornell.edu/uscode/text/17/1201