News for hackers. Stuff that matters.
Re:Move myself to inactive by LeoPanthera on Saturday December 18, 2021 @11:15PM (#29611251)
For those unaware, there is an active fork called yt-dlp, that is (mostly) a drop-in replacement:

https://github.com/yt-dlp/yt-dlp

Re:Move myself to inactive by kaetemi on Sunday December 19, 2021 @12:16AM (#29611567)
Also bypasses throttling. Much faster, most of the time.
Re:Move myself to inactive by Seirdy on Sunday December 19, 2021 @03:04AM (#29612274)
The best part is the native sponsorblock integration.

It adds chapter indicators to skip the sponsored sections, opening titles, begging viewers to inflate engagement metrics (please like, comment, and subscribe!), openings of music videos that aren't the actual music, opening and closing titles/credits, etc.

Basically cuts out all the bullshit you'd otherwise have to manually skip past by spamming the arrow keys.

Re:Move myself to inactive by stavros on Sunday December 19, 2021 @05:43AM (#29612886)
It also has a "skip to highlight" button that skips to the meat of the video and saves you 98% of its time.
Re:Move myself to inactive by busymom0 on Sunday December 19, 2021 @05:52AM (#29612933)
How does it do this? Is this something the uploader of the video provides?
Re:Move myself to inactive by stavros on Sunday December 19, 2021 @05:54AM (#29612948)
It's user-provided.
Re:Move myself to inactive by jffry on Sunday December 19, 2021 @08:28AM (#29613751)
All the data is submitted by users of the browser extension (there's a thumbsup/thumbsdown system to flag bad submissions)
Re:Move myself to inactive by Seirdy on Sunday December 19, 2021 @07:56PM (#29620394)
It can also be submitted via a Newpipe fork or mpv script.
Re:Move myself to inactive by aulin on Sunday December 19, 2021 @05:09AM (#29612720)
by the way, out of curiosity, is this something content producers add to their videos or something youtube autogenerates? I've seen SmartTubeNext skipping ads and other bullshit and I was wondering where the feature came from.
Re:Move myself to inactive by sorenjan on Sunday December 19, 2021 @05:33AM (#29612831)
Sponsorblock data is added by users using the Sponsorblock browser extensions.

https://addons.mozilla.org/sv-SE/firefox/addon/sponsorblock/

Re:Move myself to inactive by DQDPm64JTNB2 on Sunday December 19, 2021 @05:15AM (#29612754)
Sponsorblock works by it's users marking parts of videos and then submitting them. Other users then can vote if these parts are right or not.
Re:Move myself to inactive by nextaccountic on Sunday December 19, 2021 @12:44AM (#29611689)
Sounds like a great recipe for getting banned from services. But, as long as it works, I'm all for it..
Re:Move myself to inactive by alpaca128 on Sunday December 19, 2021 @03:02AM (#29612262)
It's the only way to be sure I'll ever find certain videos again. And Google randomly bans users either way, they're not gonna stop if yt-dl disappears.
Re:Move myself to inactive by wintermutestwin on Sunday December 19, 2021 @04:18AM (#29612523)
How would Google associate a YT download with a specific YouTube user?
Re:Move myself to inactive by exikyut on Sunday December 19, 2021 @06:04AM (#29613011)
I puzzled over this for a minute as well. Maybe the GP is referring to the user who uploaded the video.
Re:Move myself to inactive by kataklasm on Sunday December 19, 2021 @05:41AM (#29612883)
Why would they do that when they can just make sure yt-dlp/youtube-dl doesn't work all together. :) No need to ban users who download content, just ban the service itself (ban as in build roadblocks to deny the downloader service access to your content).
Re:Move myself to inactive by busymom0 on Sunday December 19, 2021 @05:54AM (#29612950)
They have been trying to break YouTube-dl for many years (and similar framework for iOS) and the code maintainers just need to update their code slightly to find another workaround and get it working again.
Re:Move myself to inactive by solarkraft on Sunday December 19, 2021 @07:00AM (#29613264)
For now. I can see actual DRM coming some time in the not too far future.
Re:Move myself to inactive by lowwave on Sunday December 19, 2021 @08:44AM (#29613870)
No DRM needed. many sites just use hls.js type of chunk download tactic, it is already enough to prevent ytdl from downloading.
Re:Move myself to inactive by skinnymuch on Sunday December 19, 2021 @12:52PM (#29616285)
Is there any way to solve for that? I’ve noticed that and maybe that’s it or other chunked download stuff. That I haven’t been able to download. Never looked into this stuff enough to see what solutions are (besides brute force screen recording Ofc)
Re:Move myself to inactive by no_time on Sunday December 19, 2021 @02:55PM (#29617669)
No one size fits all solution. You have to reverse engineer the way the script fetches and processes chunks and write a downloader or a plugin that instruments the running code and spits out the video data.

When wasm released I speculated that it will make this much more widespread and harder to reverse engineer due having more performace to waste on de-obfuscating at runtime. Feels nice to be wrong in this case.

Re:Move myself to inactive by solarkraft on Sunday December 19, 2021 @01:28PM (#29616710)
Oh, I thought ytdl could deal with that! Do you know the specific issue and whether support is planned? Lack of HLS support would explain why there's no support for Zoom recordings.
Re:Move myself to inactive by dannyw on Sunday December 19, 2021 @02:43AM (#29612179)
YouTube already specifically seeks to throttle youtuble-dl to an unusably slow speed, like 70kbps even if you only download one video a day.

What else are you supposed to do?

Re:Move myself to inactive by busymom0 on Sunday December 19, 2021 @05:55AM (#29612962)
Use yt-dlp which is a well maintained alternative to YouTube-dl and doesn’t have the throttling issue. All commands are the same.
Re:Move myself to inactive by crtasm on Sunday December 19, 2021 @08:00AM (#29613556)
Except for the ones that aren't, but yes it's mostly a drop in.

https://github.com/yt-dlp/yt-dlp#differences-in-default-beha...

Re:Move myself to inactive by countmora on Sunday December 19, 2021 @03:01AM (#29612255)
I always managed to get getter speeds using the aria2 parameter.
Re:Move myself to inactive by hdjjhhvvhga on Sunday December 19, 2021 @05:00AM (#29612690)
> Sounds like a great recipe for getting banned from services.

If you are using your account for downloading, that is. I never use any valuable account when dealing with automated tools, it's just asking for trouble.

Re:Move myself to inactive by kaetemi on Sunday December 19, 2021 @02:15AM (#29612080)
Apparently the trick is to pretend being an Android device, or something in that style...
Re:Move myself to inactive by varenc on Sunday December 19, 2021 @03:29AM (#29612358)
The work needed to bypass the rate limiting is pretty impressive: https://github.com/ytdl-org/youtube-dl/pull/30184#issuecomme...

An excerpt:

> ...the challenge algorithm is served in a mini-language within the minified player JS and therefore the specific algorithm could be extracted and executed by interpreting the mini-language without actually running the JS itself.

Re:Move myself to inactive by skytreader on Sunday December 19, 2021 @08:05AM (#29613598)
Quicker explanation of how Youtube is doing it is at the source code of pytube: https://github.com/tfdahlin/pytube/blob/bb890af3b271a616bbe1...

This is linked from the description of PR 30184 of youtube-dl but @varenc linked to a specific comment (talking about patching Windows builds, AFAICT) which might confuse some people.

I'll even save you the click:

> YouTube's strategy to restrict downloading videos is to send a ciphered version of the signature to the client, along with the decryption algorithm obfuscated in JavaScript. For the clients to play the videos, JavaScript must take the ciphered version, cycle it through a series of "transform functions," and then signs the media URL with the output.

There are still a lot of things I don't understand here but gotta give Youtube a slow-clap moment for their devious ingenuity.

Re:Move myself to inactive by Kadin on Sunday December 19, 2021 @11:58AM (#29615612)
> There are still a lot of things I don't understand here but gotta give Youtube a slow-clap moment for their devious ingenuity.

Well, sorta. They have basically come up with a DRM scheme by another name. They're always just speed bumps.

Re:Move myself to inactive by varenc on Sunday December 19, 2021 @02:43PM (#29617530)
Whoops, didn’t mean to include the link to that comment... that’s a much better explanation!
Re:Move myself to inactive by rstuart4133 on Sunday December 19, 2021 @05:42PM (#29619268)
As someone else said here on HN, in situations like this all it amounts to "mutually assured time wasting".

It reminded me ofthe old joke about two people entering an area infested by lion's, and one of them sits down on a log, and takes 5 minutes to replace his robust hiking shoes by a very expensive pair of running shows. The other says "why are you doing that - you can't our run a lion." The reply is "I don't have to be faster than the lion - just faster than you".

It makes prefect sense of course, when the lion has a choice. But in this case there is no choice. There is only one youtube. No matter how much time and expense youtube putting into their running shoes, youtube-dl / yt-dlp is always going to chase after them.

They are always going to win that race. While Google may be paying 5 or 10 of the cream of worlds programmers, those two open source projects ultimately how pools of 100's of equally talented people who occasionally want to download a video, and will happily contribute back a few hours of their time to do it. Which is probably how yt-dlp came about - youtube-dl slowed down, and someone said "fuck it - I'll do it myself". Worse from google's point of view, unlike splitinering a commercial endeavours, once those two projects are already contributing code to each other and may well merge again.

Re:Move myself to inactive by kevin_thibedeau on Sunday December 19, 2021 @03:18AM (#29612322)
You used to be able to bypass throttling by watching the video in browser during the download.
Re:Move myself to inactive by usr1106 on Sunday December 19, 2021 @05:35AM (#29612847)
I have always wondered why video sites (who don't want downloaders for various reasons) serve their content faster than required for normal speed viewing. Well, I don't complain, but I wonder.
Re:Move myself to inactive by gattilorenz on Sunday December 19, 2021 @05:39AM (#29612868)
For buffering, since there might be a network hiccup sometime in the future. But generally also a better user experience if the user wants to skip a few seconds/minutes forward.

And I guess the bandwidth required also depends on the video bit rate/format/compression ratio, so you need to implement a lot of smarts to make sure you don't serve something too slow or too fast, and still people would be able to download the video, albeit at a slower speed. So, basically, it's not worth it.

Re:Move myself to inactive by foxfluff on Sunday December 19, 2021 @06:44AM (#29613190)
The primary reason I've used ytdl is to work around youtube's crappy buffering.

In the past you used to be able to leave a youtube page open and it'd buffer the whole video but now it stops at like.. thirty seconds? Have fun watching when their servers can't serve.

Re:Move myself to inactive by varenc on Sunday December 19, 2021 @05:37PM (#29619219)
They address this by having the video player stop buffering more than 30 seconds ahead. It's also less compute resources for the servers to send all the bytes for the next 30 seconds to a client at max speed, let the client disconnect and wait 3-5 seconds, then give them the next 3-5 seconds immediately when it's requested. If they only let you download video at a rate slightly faster than video playback rate than all clients would have a constantly open connection downloading video which would tie up more resources.
Re:Move myself to inactive by kaetemi on Sunday December 19, 2021 @07:13AM (#29613320)
Video streams are just a series of 2-second-long video files. It's no different from any regular CDN.

(The real money is in the compression processes.)

Re:Move myself to inactive by cube00 on Sunday December 19, 2021 @08:36AM (#29613810)
I was surprised by just how many small chunks (as you said 2 seconds, a few hundred kilobytes) the browser has to request and string together to make a video stream that play smoothly and that I can skip around. It's kinda amazing when you think about it.
Re:Move myself to inactive by codetrotter on Sunday December 19, 2021 @01:18AM (#29611865)
yt-dlp works great. Worth to note that for macOS you don’t need their tap to install it, even though the yt-dlp readme says to use the tap.

yt-dlp is in the main package repository of homebrew, and the version in the main repo of homebrew is kept close to just as up to date as the tap.

So to install yt-dlp on a Mac that has homebrew installed, just run:

    brew install yt-dlp
Re:Move myself to inactive by walrus01 on Sunday December 19, 2021 @12:12AM (#29611542)
switched to it a couple of weeks back, working flawlessly for my needs.
Re:Move myself to inactive by amelius on Sunday December 19, 2021 @05:47AM (#29612906)
Do you run it inside a sandbox?
Re:Move myself to inactive by busymom0 on Sunday December 19, 2021 @05:51AM (#29612927)
I use mine via terminal the same way as YouTube-dl. The commands are the same too.