Not OP, but I too wrote something nearly identical with whisper so I could creep on old EthosLab videos. Here's the gist:

from pytube import Channel

import whisper

channel_yt = Channel(channel_url)

video_yt = channel_yt.videos[0]

video_yt_stream = video_yt.streams.filter(mime_type="video/mp4").filter(res="720p").first()

video_yt_video_file_path = video_yt_stream.download()

audio = whisper.load_audio(video_yt_video_file_path)

model = whisper.load_model("tiny")

transcript = model.transcribe(audio)