If the license between the RIAA (or other licensor) and Youtube allowed visitors to make permanent copies of Youtube streams, then downloading a Youtube video would not be a copyright violation. But generally, having entered into streaming and downloading licensing agreements with the RIAA before, I would be extremely surprised if the RIAA would enter into such a license with Youtube, as it would kill sales of digital music downloads if people could just download permanent copies of Youtube videos instead for free.
If you own the copyright and upload a video to Youtube, then it wouldn't be a copyright violation to download that video from Youtube.
Owning the copyright means you don't violate the copyright in making a copy, but that doesn't mean you get to make that copy through any means; you still have to use legal means, and for a website, that means methods that are acceptable under the TOS of that website.
YouTube is not the copyright owner (in almost all cases). Downloading a video with youtube-dl may not be a copyright violation if you already separately hold a suitable license, or are in fact the copyright owner yourself (as in the case of one of the grandparent comments).