Composite and Swap — Getting it Right

Where the author tries to make sure DRI3000 is going to do what we want now and in the future

DRI3000

The basic DRI3000 plan seems pretty straightforward:

  1. Have applications allocate buffers full of new window contents, attach pixmap IDs to those buffers and pass them to the X server to get them onto the screen.

  2. Provide a mechanism to let applications know when those pixmaps are idle so that they can reuse them instead of creating new ones for every frame.

  3. Finally, allow the actual presentation of the contents to be scheduled for a suitable time in the future, generally synchronized with the monitor. Let the client know when this has happened in case they want to synchronize themselves to vblank.

The DRI3 extension provides a way to associate pixmap IDs and buffers, and given the MIT-SHM prototype I've already implemented, I think we can safely mark this part as demonstrably implementable.

That leaves us with a smaller problem, that of taking pixmap contents and presenting them on the screen at a suitable time and telling applications about the progress of that activity.

In the absence of compositing, I'm pretty sure the initial Swap extension design would do this job just fine, and should resolve some of the known DRI2 limitations related to buffer management. And, I think that goal is sufficient motivation to go and implement that. However, I wanted to write up some further ideas to see if the DRI3000 plan can be made to do precisely what we want in a composited world.

The Composited Goal

To make sure we're all on the same page, here's what I expect from the Swap extension in a composited world:

  1. Application calls Swap with new window pixmap

  2. Compositor hears about the new pixmap and uses that to construct a new screen pixmap

  3. Compositor calls Swap with new screen pixmap

  4. Vertical retrace happens, executing the pending swap operation

  5. Compositor hears about the swap completion for the screen

  6. Application hears about the swap completion for its window

In particular, applications should not hear that their swap operations are complete until the contents appear on the screen. This allows for applications to throttle themselves to the screen rate, either doing double or triple buffering as they choose.

I didn't add steps here indicating buffers going idle or being allocated, because I think that should all happen 'behind the scenes' from the application's perspective. Many applications won't care about the swap completion notification either, but some will and so that needs to be visible.

Redirected Swaps?

Owen Taylor suggested that one way of getting the compositor involved would be to have it somehow 'redirect' Swap operations, much like we do with window management operations today. I think that idea may be a good direction to try:

  1. Application calls Swap with new window pixmap

  2. Swap is redirected to compositor, passing along the new window pixmap

  3. Compositor constructs a new screen pixmap using the new window pixmap

  4. Compositor calls Swap on the screen and the window, passing the new screen pixmap and the new window pixmap. When the screen update occurs, the screen and the window both receive swap completion events.

This has the added benefit that the X server knows when the compositor is expecting window pixmaps to change like this -- the compositor has to explicitly request Swap redirection.

Window Pixmap Names and GEM Buffer Handles

One issue that swapping window pixmaps around like this brings up is how to manage existing names for the window pixmap. Right now, applications expect that window pixmaps will only change when the window is resized. If the Swap extension is going to actually replace the window pixmap when running with a suitable compositor, then we need to figure out what the old names will reference.

Are there non-compositor applications using NameWindowPixmap that matter to us? How about non-compositor applications using TextureFromPixmap to get a GEM handle for a window pixmap? For now, I'm very tempted to just break stuff and see who complains, but knowing what we're breaking might be nice beforehand.

Idling Pixmaps

When an application is done drawing to a window pixmap and has passed it off to the X server for presentation, we'd like for that pixmap to be automatically marked as discardable as soon as possible. This way, when memory is tight, the kernel can come steal those pages for something critical. Of course, applications may not want to let the server mark the pixmap as idle after being used, so a flag to the Swap call would be needed.

Ideally, the pixmap would become idle immediately after the pixmap contents have been extracted. In the absence of a compositor, that would probably be when the Swap operation completes. With a compositor running, we'd need explicit instruction from the compositor telling us that the window pixmap was now 'idle':

┌───
    SwapIdle
    drawable: Drawable
    pixmap: Pixmap
      ▶
└───

Furthermore, the application needs to know that the pixmap is in fact idle. I think that we'll need a synchronous X request that marks a buffer as 'no longer idle' and have that return whether the buffer was discarded while idle. It doesn't seem sufficient to use events here as the application will need to completely reconstruct the pixmap contents in this case. This reply could also contain information about precisely what contents the pixmap does contain.

┌───
    SwapReuse
    drawable: Drawable
    pixmap: Pixmap
      ▶
    valid: BOOL
    swap-hi: CARD32
    swap-lo: CARD32
└───

Pixmap Lifetimes and Triple Buffered Applications

If we redirect the Swap operation and send the original application window pixmap ID to the compositor, what happens when the application frees that pixmap before the compositor gets around to using the contents?

Surely the Compositor must handle such cases, and not just crash. However, I'm fine with requiring that the application not free the pixmap until told by the compositor.