Mirror Dual Displays of Differing Resolutions on i.MX8M-PLUS

For most of our work on embedded devices, the setups are typically quite static: the main board, a display with known resolution, some sensors. But sometimes there is a requirement that shakes up the status quo. On one recent project, that was definitely the case. The customer said  “We have an LVDS display, but we want the user to be able to plug in an HDMI monitor at any time and provide a mirrored display of the application.”

Though the request was straightforward, it was not simple. Here’s why.

Back in the days of the X server accomplishing this was not a big deal. Just create the proper configuration and the X server would mirror the main display on the newly plugged display.

It would look like this:

 

But these days, achieving this is a little tricky. So what happens if the second monitor cannot handle the same resolution as the primary monitor one? You’d see something like this:

 

Things start to look bad, don’t they? To fix it, we need to use the native resolution of each monitor and adjust the window’s content in some smart way. Doing so could achieve something like this:

 

So how do we get there? Here’s our thought process and our solution.

Goodbye X, Hello Wayland

The NXP i.MX8 family of processors now favor Wayland over X. Wayland in itself is a protocol to be used between a compositor and one or more clients. Both pieces of the puzzle can be coded. The compositor’s reference implementation is named Weston. But if you are using Qt like we often do at ICS you might want to consider the Qt Wayland Compositor.

The default Wayland specification does not provide a way for clients to specify where they will be positioned. This is intentional: it’s the duty of the windows manager to handle this. And the window manager is part of the composer.

The Wayland protocol can be expanded. An example of that is the In-Vehicle Infotainment (IVI) extension. The IVI protocol allows clients to provide an ID so that the composer knows where to place the window. This concept makes a lot of sense in vehicles, letting the composer keep important windows in the right place.

Equipped with this information, we could think of creating two windows, each with its own ID, and let the composer put them in the right place. Sounds straightforward, right? We are using Qt, so let’s first try with the Qt IVI Compositor. 

Darn. That’s not going to work because part of the requirement specified that the user will plug in an HDMI monitor – but did not define which monitor, or more importantly, at what resolution. 

At the time of this writing Qt can only handle multiple displays when their resolutions are hardcoded in a configuration file. So we need to use another compositor. Weston does have an IVI extension. But it seems that, at least by default, it only handles layouts on the primary screen. So, in order to use the IVI shell we would need to modify the composer. Definitely doable, but that will take extra time.

A different option could be using the Sway compositor. Sway is a tiling Wayland compositor. Coupled to it, instead of letting the application create two windows, we could use wl-mirror.  I did not mention this before, but I am using Yocto on my Variscite IMX8M-PLUS board. So, I could bring in Sway and try to make it work.

But, there has to be an easier approach!

One Window to Handle Everything

Weston can handle multiple displays, and every time a new display is plugged in it will expand the desktop to the right of the original LVDS display, like this:

So, what if we create a single window that has as width the sum of the LVDS and HDMI displays and the height of the taller display? Then our application could show the same content on both the LVDS and HDMI areas, interacting in the same way.

Would this work? Turns out, this works amazingly well! With this approach, we can let Qt tell us the sizes of the screens and react accordingly, and it can be done quite quickly. The only drawback is that this application will request a memory area bigger than the sum of both displays, but at least we won’t need to handle the compositor’s code.

Sometimes, Simple is Best

This was one of those cases in which it was really fun to find a solution for. Sometimes one has all the tools available to do a “proper” solution – in our case, handling multiple displays and putting windows in the right places on the compositor itself – but a simple solution can be faster and just as effective.

If you enjoyed this blog, read more from Lisandro here.