News for hackers. Stuff that matters.
Re:Apple's CVDisplayLink Doesn't Link to Your Display by Someone on Sunday December 10, 2017 @06:08AM (#15890910)
The documentation (https://developer.apple.com/library/content/documentation/Gr...) doesn’t mention vertical refresh interrupts.

It also says ”In the past, synchronizing your video frames with the display’s refresh rate was often a problem, especially if you also had audio. You could only make simple guesses for when to output a frame […] which didn’t take into account possible latency from user interactions, CPU loading, window compositing and so on. The Core Video display link can make intelligent estimates for when a frame needs to be output, based on display type and latencies. […] If for some reason the processing takes longer than expected (that is, the display link’s estimate is off), the video graphics card can still drop frames or otherwise compensate for the timing error as necessary.”.

So, disassembling the code wasn’t strictly necessary.

Combining this with this article, it seems they try to call your program as late as possible, possibly to decrease latency between the time you have influence on what is displayed and the time it is displayed. Could that especially if you also had audio be the reason they do that, as it might make it easier to keep audio in sync with video?

Re:Apple's CVDisplayLink Doesn't Link to Your Display by jmull on Sunday December 10, 2017 @01:25PM (#15892682)
> it seems they try to call your program as late as possible

Right. The point of the API is to trigger rendering as late as possible before the next frame swap, not immmediarely after the last frame swap, as the author expected.

To do so, it has to estimate how long your app’s callback will take to render a frame. (E.g. if it expects your callback to render a frame in 10ms it might call it 11ms before the next frame swap.)

It seems the heuristic it uses to estimate the duration of the callback execution is assuming a fairly constant duration. The documentation seemed to suggest that. But as the author has inconsistent rendering duration.

So, e.g., if it estimates 10ms and therefore calls back 11ms before the frame swap but you end up taking 20ms, you’re going to drop a frame. If it then adjusts and calls back 21ms before, but you render in 1ms, you could render two frames in the space of one displayed frame.

Anyway, just thinking through it aloud.

I think the author may not want to use this API for his purpose, though perhaps by adjusting his rendering so that it always takes the “max” rendering time might let its estimates be correct and then things would smooth out.

Re:Apple's CVDisplayLink Doesn't Link to Your Display by trishume on Sunday December 10, 2017 @03:37PM (#15893368)
Author here, that would make sense, except it doesn't appear to be doing that. I also don't see anything in the docs suggesting that it would. My test app takes 1ms to render yet, and the callback returns within 100us, but it's consistently calling me 7ms before vsync.

I'm also not actually working on anything that uses this right now, I was just curious. I'm just warning about the dangers if you have inconsistent rendering time.

Re:Apple's CVDisplayLink Doesn't Link to Your Display by seandougall on Sunday December 10, 2017 @12:02PM (#15892250)
The callback gets a timestamp as an argument, so audio sync is easy to compensate for if you’re doing it systematically. (Plus 1/60 second is well within human perception, when it comes to audio/video synchronization.)

I think the bigger concern is latency between HID input and visual feedback, which people are more sensitive to, and which you can’t schedule in advance for.

Re:Apple's CVDisplayLink Doesn't Link to Your Display by wtallis on Sunday December 10, 2017 @08:26AM (#15891282)
Audio synchronization is a problem if the graphics pipeline has multi-frame latency, but when you're working to reduce sub-frame input latency by delaying the start of rendering, I think audio synchronization problems are by that point a minor concern that few people could notice.
Re:Apple's CVDisplayLink Doesn't Link to Your Display by stmw on Sunday December 10, 2017 @12:32PM (#15892417)
Here is some more interesting info - perhaps this is when this took place?

http://powerpcliberation.blogspot.com/2013/04/os-x-disable-y...

Re:Apple's CVDisplayLink Doesn't Link to Your Display by trishume on Sunday December 10, 2017 @11:32AM (#15892110)
Yah I had noticed their docs never said it called you after vsync.

It's not that it calls you as late as possible though, as my experiments show it just calls you at an arbitrary but consistent point in the frame like a timer.