RSS Add a new post titled:

AltOS 1.6.3 —

Bdale and I are pleased to announce the release of AltOS version 1.6.3.

AltOS is the core of the software for all of the Altus Metrum products. It consists of firmware for our cc1111, STM32L151, STMF042, LPC11U14 and ATtiny85 based electronics and Java-based ground station software.

Version 1.6.3 adds idle mode to AltosDroid and has bug fixes for our host software on desktops, laptops an android devices along with BlueTooth support for Windows.

1.6.3 is in Beta test for Android; if you want to use the beta version, join the AltosDroid beta program


AltOS fixes:

  • Fix hardware flow control on TeleBT v3.0. RTS/CTS is wired backwards on this board, switch from using the hardware to driving these pins with software.

AltosUI and TeleGPS Applications

AltosUI and TeleGPS New Features:

  • Add BlueTooth support for Windows operating system. This supports connections to TeleBT over BlueTooth rather than just USB.

AltosUI and TeleGPS Fixes:

  • Change Java detection and install on Windows. Detection is now done by looking for the 'javaw.exe' program, and installation by opening a browser on the web site.

  • Delay polling while the Fire Igniters is visible to allow for TeleMega to report back complete status over the radio.

  • Disallow changing RF calibration numbers in the configuration UI. There's no good reason to change this from the field, and recovering is really hard if you haven't written down the right number.

  • Fix USB device discovery on Mac OS X El Capitan. This makes the connected Altus Metrum USB devices appear again.

  • Fix acceleration data presented in MonitorIdle mode for TeleMetrum v2.0 flight computers.


AltosDroid new features:

  • Monitor Idle mode. Check state of flight computer while in idle mode over the radio link

  • Fire Igniters. Remotely fire ignires for recovery system ground tests.

  • Remote reboot. Cause the flight computer to reboot over the radio link. This provides a method for switching the flight computer from idle to flight mode without needing to reach the power switch.

  • Configurable frequency menu. Change the set of available frequencies and provide more descriptive names.

AltosDroid bug fixes:

  • Don't set target location if GPS hasn't locked yet.

  • Fix saving target states so they can be reloaded when the application restarts. When the application is shut down and restarted, all previous target state information will be restored (including GPS position if available).

  • Fix crash on some Android devices for offline maps when changing the map scale or location.

  • Don't require USB OTG support. This kept the latest AltosDroid from being offered on devices without USB device support, although it can work without that just fine using BlueTooth.

  • Don't require bluetooth to be enabled. This allows the application to operate with USB devices or just show old data without turning on the bluetooth radio.

  • Recover old tracker positions when restarting application. This finally allows you to safely stop and restart the application without losing the last known location of any tracker.


  • Document TeleMega and EasyMega additional pyro channel continuity audio alert pattern.
Posted Fri May 6 18:18:07 2016 Tags: Election Time — Vote Now

It's more important than usual to actually get your vote in — we're asking the membership to vote on changes the the bylaws that are necessary for to become a SPI affiliate project, instead of continuing on as a separate organization. While I'm in favor of this transition as I think it will provide much needed legal and financial help, the real reason we need everyone to vote is that we need ⅔ of the membership to cast ballots for the vote to be valid. Last time, we didn't reach that value, so even though we had a majority voting in favor of the change, it didn't take effect. If you aren't in favor of this change, I'd still encourage you to vote as I'd like to get a valid result, no matter the outcome.

Of course, we're also electing four members to the board. I'm happy to note that there are five candidates running for the four available seats, which shows that there are enough people willing to help serve the community in this fashion.

Posted Tue Apr 12 23:51:37 2016 Tags:

Automatic Calendar Management — Notmuch + Calypso

One of the big “features” of outlook/exchange in my world is automatically merging of incoming calendar updates from email. This makes my calendar actually useful in knowing what meetings people have asked me to attend. As I'm not willing to otherwise tolerate outlook, I decided to try and provide that in my preferred environment; notmuch and calypso.

Identifying calendar updates

The first trick is how to identify incoming messages with calendar updates. I'd love to be able to search for specific mime content types, but I haven't been able to construct such a search. Failing that, I'm just looking for messages containing the string 'text/calendar':

notmuch search --output=messages tag:inbox AND text/calendar

Next, I want to skip over previously scanned calendar updates, so I'll plan on tagging messages that have been scanned with the 'calendar' tag and skip those:

notmuch search --output=messages tag:inbox AND text/calendar AND not tag:calendar

jq — sed for json

With the messages containing likely calendar entries identified, the remaining task is to extract the MIME section containing the actual calendar data. Notmuch can generate json for the message, leaving us only needing to parse the json and extract the relevant section. I found the 'jq' tool in the archive, which looks like a rather complicated parsing and reformatting tool for json data. It took a while to understand, but I managed to generate a command string that takes a notmuch message and pulls out the content for all text/calendar elements:

jq -r '..| select(."content-type"? == "text/calendar") | .content'

This is a recursive walk over the data structure. It looks for structures with "content-type": "text/calendar" and dumps their "content" elements in raw text form.

Putting it all together

Here's the complete script:


SEARCH="tag:inbox AND not tag:calendar AND text/calendar"


trap "rm -r $TMP" 0 1 15

notmuch search --output=messages $SEARCH | while read message; do
    notmuch show --format=json $message | 
        jq -r '.. | select(."content-type"? == "text/calendar") | .content' > $TMP
    if [ -s $TMP ]; then
        calypso --import private/calendar $TMP && notmuch tag +calendar $message
        notmuch tag +calendar $message

I'd love to fix calypso's --import operation to talk to the local calypso daemon with the database loaded; the current mechanism actually loads the entire database into a new process and then applies the new data to that. With my calendar often containing hundreds of entries, that takes a while.

Posted Wed Jan 13 14:01:42 2016

AltOS 1.6.2 — TeleMega v2.0 support, bug fixes and documentation updates

Bdale and I are pleased to announce the release of AltOS version 1.6.2.

AltOS is the core of the software for all of the Altus Metrum products. It consists of firmware for our cc1111, STM32L151, STMF042, LPC11U14 and ATtiny85 based electronics and Java-based ground station software.

This is a minor release of AltOS, including support for our new TeleMega v2.0 board, a small selection of bug fixes and a major update of the documentation

AltOS Firmware — TeleMega v2.0 added

The updated six-channel flight computer, TeleMega v2.0, has a few changes from the v1.0 design:

  • CC1200 radio chip instead of the CC1120. Better receive performance for packet mode, same transmit performance.

  • Serial external connector replaced with four PWM channels for external servos.

  • Companion pins rewired to match EasyMega functionality.

None of these change the basic functionality of the device, but they do change the firmware a bit so there's a new package.

AltOS Bug Fixes

We also worked around a ground station limitation in the firmware:

  • Slow down telemetry packets so receivers can keep up. With TeleMega v2 offering a fast CPU and faster radio chip, it was overrunning our receivers so a small gap was introduced between packets.

AltosUI and TeleGPS applications

A few minor new features are in this release

  • Post-flight TeleMega and EasyMega orientation computations were off by a factor of two

  • Downloading eeprom data from flight hardware would bail if there was an error in a data record. Now it keeps going.


I spent a good number of hours completely reformatting and restructuring the Altus Metrum documentation.

  • I've changed the source format from raw docbook to asciidoc, which has made it much easier to edit and to use docbook features like links.

  • The css moves the table of contents out to a sidebar so you can navigate the html format easily.

  • There's a separate EasyMini manual now, constructed by taking sections from the larger manual.

Posted Sun Jan 10 21:03:08 2016 Tags:

TeleLaunchTwo — A Smaller Wireless Launch Controller

I've built a wireless launch control system for NAR and OROC. Those are both complex systems with a single controller capable of running hundreds of pads. And, it's also complicated to build, with each board hand-made by elves in our Portland facility (aka, my office).

A bunch of people have asked for something simpler, but using the same AES-secured two-way wireless communications link, so I decided to just build something and see if we couldn't eventually come up with something useful. I think if there's enough interest, I can get some boards built for reasonable money.

Here's a picture of the system; you can see the LCO end in a box behind the pad end sitting on the bench.

Radio Link

Each end has a 35mW 70cm digital transceiver (so, they run in the 440MHz amateur band). These run at 19200 baud with fancy forward error correction and AES security to keep the link from accidentally (or maliciously) firing a rocket at the wrong time. Using a bi-directional link, we also get igniter continuity and remote arming information at the LCO end.

The LCO Box

In the LCO box, there's a lipo battery to run the device, so it can be completely stand-alone. It has three switches and a button -- an arming switch for each of two channels, a power switch and a firing button. The lipo can be charged by opening up the box and plugging it into a USB port.

The Pad Box

The pad box will have some cable glands for the battery and each firing circuit. On top, it will have two switches, a power switch and an arming switch. The board has two high-power FETs to drive the igniters. That should be more reliable than using a relay, while also allowing the board to tolerate a wider range of voltages -- the pad box can run on anything from 12V to 24V.

The Box

Unlike the OROC and NAR systems, these boards are both designed to fit inside a specific box, the Hammond 1554E, and use the mounting standoffs provided. This box is rated at NEMA 4X, which means it's fairly weather proof. Of course, I have to cut holes in the box, but I found some NEMA 4X switches, will use cable glands for the pad box wiring and can use silicone around the BNC connector. The result should be pretty robust. I also found a pretty solid-seeming BNC connector, which hooks around the edge of the board and also clips on to the board.

Safety Features.

There's an arming switch on both ends of the link, and you can't fire a rocket without having both ends armed. That provides an extra measure of safety while working near the pad. The pad switch is a physical interlock between the power supply and the igniters, so even if the software is hacked or broken, disarming the box means the igniters won't fire.

The LCO box beeps constantly when either arming switch is selected, giving you feedback that the system is ready to fire. And you can see on any LED whether the pad box is also armed.

Posted Sun Dec 20 19:51:38 2015 Tags:

The Machine Architecture

Here's a brief introduction to some of the hardware concepts within The Machine.

As the team at HP that I'm working with are busy working on Linux kernel changes motivated by the hardware, I'm hoping that providing this kind of documentation will help Linux kernel developers outside of HP evaluate that work, and work by others in related areas.

Joining HP and learning about The Machine

In January, I joined HP to work on Linux for The Machine.

I'd watched Martin Fink's video and read other articles on the new hardware coming out of HP labs. I had hints of what they were up to, and the possibilities seemed exciting enough to entice me to go back to HP.

When I arrived at HP, one of the first things I got to read was the external reference specification for The Machine. 170 pages detailing a more significant shift in computer architecture than I had been given any hints of, both in my interviews at HP and from what I could see in the press.

Since then, I've been eager to tell people about what we're doing, and I'm happy to say that we're finally ready to start the conversation with this brief description.

A Short Outline of Storage within The Machine

The basic unit of The Machine is a collection of hardware grouped in a Load Store Domain. A Load Store Domain consists of:

  • Multiple independent Compute Nodes

    • Independent operating systems
    • Local memory
    • Load/store access to shared, in-memory storage
  • Shared byte-addressable persistent memory:

    • Non-volatile wrt operating system life cycle
    • Global address space
    • Hardware access control
    • Accessed with standard CPU load/store instructions

Here's a diagram of how the various bits of the hardware are hooked together:

And, a brief description of the elements within the picture:

  • Compute Node. A set of processing cores, caches and various ancillary peripherals.
  • Local Store. Memory directly connected to the processing cores.
  • Firewall. Hardware access control between the compute node and shared memory.
  • Shared Byte-addressable Persistent Memory. This is the storage within The Machine. It is accessed directly via normal CPU load/store instructions in units as small as one byte.

I've intentionally drawn the shared memory in a large box to emphasize the notion that this machine is more "memory-centric" and less "processor-centric".

The shared byte-addressable persistent memory forms the sole persistent storage within The Machine.

More to Come

I'll continue to publish information about The Machine and our related Linux work as we work on the hardware and software.

Posted Thu Apr 30 22:44:42 2015 Tags:

AltOS 1.6 — TeleDongle v3.0 support and bug fixes

Bdale and I are pleased to announce the release of AltOS version 1.6.

AltOS is the core of the software for all of the Altus Metrum products. It consists of firmware for our cc1111, STM32L151, STMF042, LPC11U14 and ATtiny85 based electronics and Java-based ground station software.

This is a major release of AltOS, including support for our new TeleDongle v3.0 board and a selection of bug fixes

AltOS Firmware — TeleDongle v3.0 added along with some fixes

Our updated ground station, TeleDongle v3.0, works just like the original TeleDongle, but is an all-new design:

  • CC1200 radio chip is about 5dB more sensitive than TeleDongle's CC1111.

  • LPC11U14 CPU can be reprogrammed over the USB link.

AltOS Bug Fixes

We also fixed a few bugs in the firmware:

  • Make sure the startup flight computer beeps are consistent. Sometimes, it was taking long enough to beep out the battery voltage that the flight computer state was changing in the middle, causing a bit of confusion.

  • Change TeleDongle's LED indicators during telemetry reception. The green LED blinks on successful packet reception, and the red LED blinks when a packet with an invalid checksum is received.

  • The SPI driver used in both TeleDongle v3 and TeleGPS has been rewritten to avoid locking up under heavy CPU load. If you've got a TeleGPS board, you'll want to reflash with new firmware.

AltosUI and TeleGPS applications

A few minor new features are in this release

  • AltosUI can now compute and display tilt angle when graphing eeprom log data from TeleMega and EasyMega.

  • The AltosUI tool window is shown when starting with a data file. This way, when you double-click on a file in the file manager, you'll get the whole AltosUI interface, rather than just the graphing window.

  • At the end of replaying an old log file, stick 'done' in the Age field so you can tell the recording is over.

Bug Fixes

There are a bunch of minor bug fixes, including the usual collection of attempts to make stuff behave better on Windows platforms.

  • Use a different Windows API to discover USB device ids. This works better on my new HP Windows 7 machine. Maybe it will work better for other people too?

  • Look in more places in the Windows registry to try and find the installed Java version. It appears that the default Java download from Oracle is a 32-bit version? In any case, that version sticks its install information in a different spot in the registry.

  • Fix file associations on Windows when Java isn't installed in the system root.

  • Make 'Scan Channels' work better with new AltOS firmware which only reports device configuration information once ever five seconds.

Posted Sat Feb 7 23:57:15 2015 Tags:

Re-joining HP

Thursday was my first day back with HP. I've joined Steve Geary's group to work on Linux support for “the machine”

I had a great time at Intel and wish my old team all the best.

Posted Fri Jan 9 18:17:25 2015 Tags: ?tags/debian

Multi-Stream Transport 4k Monitors and X

I'm sure you've seen a 4k monitor on a friends desk running Mac OS X or Windows and are all ready to go get one so that you can use it under Linux.

Once you've managed to acquire one, I'm afraid you'll discover that when you plug it in, you're limited to 30Hz refresh rates at the full size, unless you're running a kernel that is version 3.17 or later. And then...

Good Grief! What Is My Computer Doing!

Ok, so now you're running version 3.17 and when X starts up, it's like you're using a gigantic version of Google Cardboard. Two copies of a very tall, but very narrow screen greets you.

Welcome to MST island.

In order to drive these giant new panels at full speed, there isn't enough bandwidth in the display hardware to individually paint each pixel once during each frame. So, like all good hardware engineers, they invented a clever hack.

This clever hack paints the screen in parallel. I'm assuming that they've got two bits of display hardware, each one hooked up to half of the monitor. Now, each paints only half of the pixels, avoiding costly redesign of expensive silicon, at least that's my surmise.

In the olden days, if you did this, you'd end up running two monitor cables to your computer, and potentially even having two video cards. Today, thanks to the magic of Display Port Multi-Stream Transport, we don't need all of that; instead, MST allows us to pack multiple cables-worth of data into a single cable.

I doubt the inventors of MST intended it to be used to split a single LCD panel into multiple "monitors", but hardware engineers are clever folk and are more than capable of abusing standards like this when it serves to save a buck.

Turning Two Back Into One

We've got lots of APIs that expose monitor information in the system, and across which we might be able to wave our magic abstraction wand to fix this:

  1. The KMS API. This is the kernel interface which is used by all graphics stuff, including user-space applications and the frame buffer console. Solve the problem here and it works everywhere automatically.

  2. The libdrm API. This is just the KMS ioctls wrapped in a simple C library. Fixing things here wouldn't make fbcons work, but would at least get all of the window systems working.

  3. Every 2D X driver. (Yeah, we're trying to replace all of these with the one true X driver). Fixing the problem here would mean that all X desktops would work. However, that's a lot of code to hack, so we'll skip this.

  4. The X server RandR code. More plausible than fixing every driver, this also makes X desktops work.

  5. The RandR library. If not in the X server itself, how about over in user space in the RandR protocol library? Well, the problem here is that we've now got two of them (Xlib and xcb), and the xcb one is auto-generated from the protocol descriptions. Not plausible.

  6. The Xinerama code in the X server. Xinerama is how we did multi-monitor stuff before RandR existed. These days, RandR provides Xinerama emulation, but we've been telling people to switch to RandR directly.

  7. Some new API. Awesome. Ok, so if we haven't fixed this in any existing API we control (kernel/libdrm/, then we effectively dump the problem into the laps of the desktop and application developers. Given how long it's taken them to adopt current RandR stuff, providing yet another complication in their lives won't make them very happy.

All Our APIs Suck

Dave Airlie merged MST support into the kernel for version 3.17 in the simplest possible fashion -- pushing the problem out to user space. I was initially vaguely tempted to go poke at it and try to fix things there, but he eventually convinced me that it just wasn't feasible.

It turns out that all of our fancy new modesetting APIs describe the hardware in more detail than any application actually cares about. In particular, we expose a huge array of hardware objects:

  • Subconnectors
  • Connectors
  • Outputs
  • Video modes
  • Crtcs
  • Encoders

Each of these objects exposes intimate details about the underlying hardware -- which of them can work together, and which cannot; what kinds of limits are there on data rates and formats; and pixel-level timing details about blanking periods and refresh rates.

To make things work, some piece of code needs to actually hook things up, and explain to the user why the configuration they want just isn't possible.

The sticking point we reached was that when an MST monitor gets plugged in, it needs two CRTCs to drive it. If one of those is already in use by some other output, there's just no way you can steal it for MST mode.

Another problem -- we expose EDID data and actual video mode timings. Our MST monitor has two EDID blocks, one for each half. They happen to describe how they're related, and how you should configure them, but if we want to hide that from the application, we'll have to pull those EDID blocks apart and construct a new one. The same goes for video modes; we'll have to construct ones for MST mode.

Every single one of our APIs exposes enough of this information to be dangerous.

Every one, except Xinerama. All it talks about is a list of rectangles, each of which represents a logical view into the desktop. Did I mention we've been encouraging people to stop using this? And that some of them listened to us? Foolishly?

Dave's Tiling Property

Dave hacked up the X server to parse the EDID strings and communicate the layout information to clients through an output property. Then he hacked up the gnome code to parse that property and build a RandR configuration that would work.

Then, he changed to RandR Xinerama code to also parse the TILE properties and to fix up the data seen by application from that.

This works well enough to get a desktop running correctly, assuming that desktop uses Xinerama to fetch this data. Alas, gtk has been "fixed" to use RandR if you have RandR version 1.3 or later. No biscuit for us today.

Adding RandR Monitors

RandR doesn't have enough data types yet, so I decided that what we wanted to do was create another one; maybe that would solve this problem.

Ok, so what clients mostly want to know is which bits of the screen are going to be stuck together and should be treated as a single unit. With current RandR, that's some of the information included in a CRTC. You pull the pixel size out of the associated mode, physical size out of the associated outputs and the position from the CRTC itself.

Most of that information is available through Xinerama too; it's just missing physical sizes and any kind of labeling to help the user understand which monitor you're talking about.

The other problem with Xinerama is that it cannot be configured by clients; the existing RandR implementation constructs the Xinerama data directly from the RandR CRTC settings. Dave's Tiling property changes edit that data to reflect the union of associated monitors as a single Xinerama rectangle.

Allowing the Xinerama data to be configured by clients would fix our 4k MST monitor problem as well as solving the longstanding video wall, WiDi and VNC troubles. All of those want to create logical monitor areas within the screen under client control

What I've done is create a new RandR datatype, the "Monitor", which is a rectangular area of the screen which defines a rectangular region of the screen. Each monitor has the following data:

  • Name. This provides some way to identify the Monitor to the user. I'm using X atoms for this as it made a bunch of things easier.

  • Primary boolean. This indicates whether the monitor is to be considered the "primary" monitor, suitable for placing toolbars and menus.

  • Pixel geometry (x, y, width, height). These locate the region within the screen and define the pixel size.

  • Physical geometry (width-in-millimeters, height-in-millimeters). These let the user know how big the pixels will appear in this region.

  • List of outputs. (I think this is the clever bit)

There are three requests to define, delete and list monitors. And that's it.

Now, we want the list of monitors to completely describe the environment, and yet we don't want existing tools to break completely. So, we need some way to automatically construct monitors from the existing RandR state while still letting the user override portions of it as needed to explain virtual or tiled outputs.

So, what I did was to let the client specify a list of outputs for each monitor. All of the CRTCs which aren't associated with an output in any client-defined monitor are then added to the list of monitors reported back to clients. That means that clients need only define monitors for things they understand, and they can leave the other bits alone and the server will do something sensible.

The second tricky bit is that if you specify an empty rectangle at 0,0 for the pixel geometry, then the server will automatically compute the geometry using the list of outputs provided. That means that if any of those outputs get disabled or reconfigured, the Monitor associated with them will appear to change as well.

Current Status

Gtk+ has been switched to use RandR for RandR versions 1.3 or later. Locally, I hacked libXrandr to override the RandR version through an environment variable, set that to 1.2 and Gtk+ happily reverts back to Xinerama and things work fine. I suspect the plan here will be to have it use the new Monitors when present as those provide the same info that it was pulling out of RandR's CRTCs.

KDE appears to still use Xinerama data for this, so it "just works".

Where's the code

As usual, all of the code for this is in a collection of git repositories in my home directory on fd.o:

git:// master
git:// master
git:// master
git:// randr-monitors

RandR protocol changes

Here's the new sections added to randrproto.txt


1.5. Introduction to version 1.5 of the extension

Version 1.5 adds monitors

 • A 'Monitor' is a rectangular subset of the screen which represents
   a coherent collection of pixels presented to the user.

 • Each Monitor is be associated with a list of outputs (which may be

 • When clients define monitors, the associated outputs are removed from
   existing Monitors. If removing the output causes the list for that
   monitor to become empty, that monitor will be deleted.

 • For active CRTCs that have no output associated with any
   client-defined Monitor, one server-defined monitor will
   automatically be defined of the first Output associated with them.

 • When defining a monitor, setting the geometry to all zeros will
   cause that monitor to dynamically track the bounding box of the
   active outputs associated with them

This new object separates the physical configuration of the hardware
from the logical subsets  the screen that applications should
consider as single viewable areas.

1.5.1. Relationship between Monitors and Xinerama

Xinerama's information now comes from the Monitors instead of directly
from the CRTCs. The Monitor marked as Primary will be listed first.


5.6. Protocol Types added in version 1.5 of the extension

          primary: BOOL
          automatic: BOOL
          x: INT16
          y: INT16
          width: CARD16
          height: CARD16
          width-in-millimeters: CARD32
          height-in-millimeters: CARD32
          outputs: LISTofOUTPUT }


7.5. Extension Requests added in version 1.5 of the extension.

    window : WINDOW
    timestamp: TIMESTAMP
    monitors: LISTofMONITORINFO
    Errors: Window

    Returns the list of Monitors for the screen containing

    'timestamp' indicates the server time when the list of
    monitors last changed.

    window : WINDOW
    Errors: Window, Output, Atom, Value

    Create a new monitor. Any existing Monitor of the same name is deleted.

    'name' must be a valid atom or an Atom error results.

    'name' must not match the name of any Output on the screen, or
    a Value error results.

    If 'info.outputs' is non-empty, and if x, y, width, height are all
    zero, then the Monitor geometry will be dynamically defined to
    be the bounding box of the geometry of the active CRTCs
    associated with them.

    If 'name' matches an existing Monitor on the screen, the
    existing one will be deleted as if RRDeleteMonitor were called.

    For each output in 'info.outputs, each one is removed from all
    pre-existing Monitors. If removing the output causes the list of
    outputs for that Monitor to become empty, then that Monitor will
    be deleted as if RRDeleteMonitor were called.

    Only one monitor per screen may be primary. If 'info.primary'
    is true, then the primary value will be set to false on all
    other monitors on the screen.

    RRSetMonitor generates a ConfigureNotify event on the root
    window of the screen.

    window : WINDOW
    name: ATOM
    Errors: Window, Atom, Value

    Deletes the named Monitor.

    'name' must be a valid atom or an Atom error results.

    'name' must match the name of a Monitor on the screen, or a
    Value error results.

    RRDeleteMonitor generates a ConfigureNotify event on the root
    window of the screen.

Posted Wed Dec 17 01:36:43 2014 Tags:

Present and Compositors

The current Present extension is pretty unfriendly to compositing managers, causing an extra frame of latency between the applications operation and the scanout buffer. Here's how I'm fixing that.

An extra frame of lag

When an application uses PresentPixmap, that operation is generally delayed until the next vblank interval. When using X without composting, this ensures that the operation will get started in the vblank interval, and, if the rendering operation is quick enough, you'll get the frame presented without any tearing.

When using a compositing manager, the operation is still delayed until the vblank interval. That means that the CopyArea and subsequent Damage event generation don't occur until the display has already started the next frame. The compositing manager receives the damage event and constructs a new frame, but it also wants to avoid tearing, so that frame won't get displayed immediately, instead it'll get delayed until the next frame, introducing the lag.

Copy now, complete later

While away from the keyboard this morning, I had a sudden idea -- what if we performed the CopyArea and generated Damage right when the PresentPixmap request was executed but delayed the PresentComplete event until vblank happened.

With the contents updated and damage delivered, the compositing manager can immediately start constructing a new scene for the upcoming frame. When that is complete, it can also use PresentPixmap (either directly or through OpenGL) to queue the screen update.

If it's fast enough, that will all happen before vblank and the application contents will actually appear at the desired time.

Now, at the appointed vblank time, the PresentComplete event will get delivered to the client, telling it that the operation has finished and that its contents are now on the screen. If the compositing manager was quick, this event won't even be a lie.

We'll be lying less often

Right now, the CopyArea, Damage and PresentComplete operations all happen after the vblank has passed. As the compositing manager delays the screen update until the next vblank, then every single PresentComplete event will have the wrong UST/MSC values in it.

With the CopyArea happening immediately, we've a pretty good chance that the compositing manager will get the application contents up on the screen at the target time. When this happens, the PresentComplete event will have the correct values in it.

How can we do better?

The only way to do better is to have the PresentComplete event generated when the compositing manager displays the frame. I've talked about how that should work, but it's a bit twisty, and will require changes in the compositing manager to report the association between their PresentPixmap request and the applications' PresentPixmap requests.

Where's the code

I've got a set of three patches, two of which restructure the existing code without changing any behavior and a final patch which adds this improvement. Comments and review are encouraged, as always!

git:// present-compositor
Posted Sat Dec 13 00:28:06 2014 Tags:

All Entries