Hardware overlay

Hardware overlay, a type of video overlay, is a method of rendering an image to a display screen with a dedicated memory buffer inside computer video hardware, to display a fast-moving video image such as a computer game, a DVD, or the signal from a TV card. Hardware overlay is supported by most video cards (since about 1998) and media players.

Overview

The use of a hardware overlay is important for several reasons. First, in a graphical user interface (GUI) operating system such as Windows, one display is typically used to display multiple simultaneous applications. Second, consider how a display works without a hardware overlay. When each application draws to the screen, the operating system's graphical subsystem must constantly check to ensure that the objects being drawn appear on the appropriate location on the screen, and that they don't collide with overlapping and neighboring windows. The graphical subsystem must clip objects while they are being drawn when a collision occurs. This constant checking and clipping ensures that different applications can cooperate with one another in sharing a display, but also consumes a significant proportion of computing power.

The way a computer draws on its display is by writing a bitmapped representation of the graphics into a special portion of its memory known as video memory. Without any hardware overlays, there is only one chunk of video memory which all applications must share - and the location of a given application's video memory moves whenever the user changes the position of the application's window. With shared video memory, an application must constantly check that it is only writing to memory that belongs to that application.

When running a high-bandwidth video application such as a game or a movie player, the computing power and complexity needed to perform this constant clipping and checking negatively impacts performance and compatibility. To escape these limitations, the hardware overlay was invented. In addition, the graphics processing unit (GPU) provides an efficient way to scale the video in size and often performs color format conversions (e.g. MPEG-2's YCbCr into RGB).

An application using a hardware overlay gets a completely separate section of video memory that belongs only to that application. Because nothing else uses it, the program never needs to waste time considering whether a given piece of the memory belongs to it, nor does it need to worry about the user moving the window and changing the location of the video memory. To get the image from the separate video memory to display in tandem with the remaining shared elements on the display, the graphical subsystem associates a certain attribute (for example, a particular color) to be a "mask" for that overlay, which the graphics card understands to mean that it is to draw from the separate overlay buffer onto the screen. (This technique is commonly known as "chroma key".)

As an example, assume the color "purple" is defined as the mask color that triggers the overlay to appear. An application that plays a DVD will draw a solid purple rectangle on the shared screen, and then "play" the DVD into the special region of memory dedicated to the overlay. The graphics card will render the DVD playback only inside the purple area. If another window moves over top of the purple area and obscures a part of it, then the graphics hardware performs the clipping by itself. In practice, the actual color "purple" is not used - rather, either a nondescript near-black color is used (as the system often has thousands of such shades to choose from), or the mask region is denoted using some similar masking method not involving colors.

creenshots

One consequence of hardware overlay use is that a screenshot program (for example, the one automatically built into Windows that activates when the PrtSc key is pressed) often does not capture the content appearing in the hardware overlay window. Rather, a blank region containing only the special mask color is captured. This is because the screen capture routine doesn't consider the special video memory regions dedicated to overlays - it simply captures the shared main screen as rendered by the software's graphical subsystem. Some Digital Rights Management schemes use hardware overlay to display protected content on the screen, taking advantage of this quirk to prevent the copying of protected documents by way of screen captureFact|date=February 2007. Disabling the support for overlays causes the normally overlay using software to fallback to the shared memory, enabling screenshots.

econdary displays

Many newer graphics cards can support more than one monitor and/or a TV screen as output device. Typically one of these output devices has to be declared the "primary" device, and only the primary device can display hardware overlays. There are exceptions: Intel writes in the FAQ for their Embedded Graphics Drivers that the overlay can be attached to either one of the displays but not both (note: the Intel 945, G33-Q965 chipsets now have dual hardware overlay, and are capable of good quality mpeg2 on secondary monitors when appropriate software is installed), and some newer Matrox graphics card support overlay on both displays (e.g. Parhelia Series).

Both hardware and driver support is required; some graphics cards may support overlay on the second display while their drivers may not yet support it (note: recent (2008.07) graphics chipset driver bugs can cause most video formats apart from mpeg2 to work on both monitors, and mpeg2 only on the primary with most players).

A common complaint is that DVD movies are displayed properly on a laptop screen but don't display on a TV connected to the laptop; in these cases it may be possible to designate the TV as the primary display. Sometimes, the use of hardware overlays may have to be disabled in the media player. However, some graphics cards have the option to completely redirect hardware overlay to the TV screen. In this case, starting a DVD player on the main screen with overlay enabled would result in video being displayed on the attached TV screen.

Implementations in various operating systems

Starting with Windows Vista's enhanced graphics capabilities, the basic concept of hardware overlays is replaced by full hardware compositing for every application window running on the system, not just movie players or games, through the Desktop Window Manager. Mac OS X has been using hardware compositing since Quartz Extreme was introduced in Mac OS X 10.2. To improve performance, each program draws to its own independent memory buffer instead of to a slow graphical subsystem. (In Windows Vista, each hardware overlay is more correctly known as a Direct3D surface). Then, the system's GPU assembles each of the windows into a single display screen in real time. With enhanced GPUs on the market capable of stunning 3D graphics as a consequence of the video game industry, impressive motion, scaling, and lighting effects can be applied to normal 2D windows by the operating system.

In the X Window System, the windowing system of most Unix operating systems, the XVideo extension can be used to allow applications to employ hardware overlays. Compositing is also used, with compiz and Beryl compositing window managers being the most prominent examples since 2006. They are able to take advantage of OpenGL (through glx extension) for 3D and 2D overlay visual effects. Other implementations like Metacity and xfwm have been available since 2004.

The Amiga supported a form of hardware compositing through the use of a coprocessor called copper. The copper ran a very simple program ("copperlist") that was synchronized with the video display hardware. The copper could execute a WAIT instruction which caused its program execution to pause until the video beam reached a particular position on the screen. Then, the copper would execute the next instruction, a MOVE instruction, which would reprogram the display hardware's memory pointer. This would cause the display hardware to immediately begin reading from a new memory location and displaying the image held there. The copper could also change the video hardware's pixel clock, allowing multiple resolutions to be displayed simultaneously.