I've wasted some time on trying to get multicast RTP working for various purposes and I don't recommend it. Software on the receiving end often isn't equipped to deal with it, so it falls flat. E.g. ffmpeg based software can connect to multicast RTP, but you need to explicitly state that you want multicast. A lot of software that uses ffmpeg doesn't even allow you to set those options.

About VNC, yes, it is slow and laggy... on a raspberry pi 3. ;)

It depends on the encoding and implementation, really. For example, I see that Mr. Wolf is using raw encoding for the RTP stream, so bandwidth is hardly a problem. Thus, I would be surprised if VNC with raw encoding would be any slower than that. In fact, it should be even faster because WayVNC will only send the parts of the image that actually changed, whereas wf-recorder will send the whole image every time. Of course, it also depends on the client. I recommend tigervnc.

I would also like to mention that VNC has, recently, gained support for H264 encoding and WayVNC can do zero-copy h264 encoding on the GPU/VPU using VAAPI. Clients that support h264 so far are: tigervnc and wlvncc. However, only wlvncc does it using hardware decoding and hardware rendering. It really falls short, when it needs to touch the CPU. When that happens, you're better off with Tight encoding.

Edit: I should mention that h264 does not work on Raspberry Pi yet. It needs hardware specific encoder and decoder.

Edit 2: Actually, I misremembered about RTP. My troubles were with multicast RTSP.