Large display format touch gesture interface

The inventive concepts disclosed herein relate generally to avionics displays installed in aircraft cockpits, and more particularly to touch-sensitive large-format displays installable aboard a variety of aircraft.

Many current avionics systems include groups of smaller-format displays (15 inches or less in diameter) without inter-display interactivity. This lack of interactivity, coupled with limited display space and cumbersome format control, requires additional workload both to configure the flight deck for a pilot and for the pilot to monitor relevant information inflight. A measure of data interactivity may be provided through the use of electronic flight bags (EFB) installed on a pad, tablet, or other portable computing device. The data interactivity provided does not rise to the level of full integration, however, and the EFB introduces new challenges. For example, in order to be accessible the EFB device may be suction-cupped to a bulkhead or window wherever space allows, or strapped to the pilot's leg. In either case, bad ergonomics and higher pilot error result. It may therefore be desirable to enhance pilot situational awareness by providing a fully integrated, large-scale flight deck avionics system with simplified and robust format control. It may further be desirable to provide an integrated flight avionics system that provides seamless interactivity with an EFB device without the need to physically interact with the EFB device itself.

In one aspect, embodiments of the inventive concepts disclosed herein are directed to an avionics display system. For example, the avionics display system may include one or more large-scale primary avionics displays for displaying a background interface such as a synthetic vision system and multiple foreground interfaces, the various interfaces responsive to finger contacts and gestures. The avionics display system may include a secondary avionics display for rehosting EFB devices with which the avionics display system links (either wirelessly or via wired connection). The secondary avionics display may relay the interfaces of EFB applications through the display surface, responding to finger contacts and gestures with the interface and relaying avionics events to the EFB via wireless link. The avionics display system may include one or more format selector panels for displaying formatting menus of the primary avionics display interfaces, the format selector panel interfaces responsive to finger contacts on its surface or on the surface of the primary or secondary avionics display. The avionics display system may include one or more keyboard panels for displaying a radio tuning interface as well as an alphanumeric keyboard interface when text entry is required, e.g., the pilot touches a text field on a primary avionics display interface.

In a further aspect, the inventive concepts disclosed herein are directed to a method for format control via an avionics display, the method including generating a response via the avionics display system by performing at least one finger contact with interfaces displayed by at least one of a primary avionics display of the avionics display system, a secondary avionics display of the avionics display system, a format selector panel of the avionics display system, and a keyboard panel of the avionics display system.

The inventive concepts disclosed herein may be better understood by those skilled in the art by reference to the accompanying figures in which:

FIG. 1 is a diagrammatic illustration of an avionics display system according to embodiments of the inventive concepts disclosed herein;

FIGS. 2A, 2B, 3A, 3B, 3C, and 3D are diagrammatic illustrations of a primary avionics display according to embodiments of the inventive concepts disclosed herein;

FIG. 4A is a diagrammatic illustration of a format selector panel according to embodiments of the inventive concepts disclosed herein;

FIG. 4B is a diagrammatic illustration of a primary avionics display interacting with a format selector panel according to embodiments of the inventive concepts disclosed herein;

FIG. 5 is a diagrammatic illustration of a secondary avionics display interacting with one or more electronic flight bag (EFB) devices according to embodiments of the inventive concepts disclosed herein;

FIG. 6A is a diagrammatic illustration of a keyboard panel according to embodiments of the inventive concepts disclosed herein;

FIG. 6B is a diagrammatic illustration of a primary avionics display interacting with a keyboard panel according to embodiments of the inventive concepts disclosed herein; and

FIG. 7 is a process flow diagram illustrating a method according to embodiments of the inventive concepts disclosed herein.

Features of the inventive concepts disclosed herein in their various embodiments are exemplified by the following descriptions with reference to the accompanying drawings, which describe the inventive concepts with further detail. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not necessarily restrictive of the inventive concepts disclosed and claimed herein. These drawings depict only exemplary embodiments of the inventive concepts, and should not be considered to limit their scope in any way.

Referring to FIG. 1 , an avionics display system 100 may be installed in an aircraft cockpit. In one embodiment, the avionics display system 100 includes multiple touch-sensitive interactive display surfaces. For example, the avionics display system 100 may include one or more primary avionics displays 102 , a secondary avionics display 104 , one or more format selector panels (FSP) 106 , and one or more keyboard panels 108 . In one embodiment, as shown by FIG. 1 , the avionics display system 100 includes a pilot-side primary avionics display 102 a , format selector panel 106 a , and keyboard panel 108 a positioned for the use of a first pilot, as well as a co-pilot-side primary avionics display 102 b , format selector panel 106 b , and keyboard panel 108 b positioned for the use of a second pilot, co-pilot, or crewmember. The primary avionics displays 102 may be large-format flat-glass immersive displays configured to display a variety of information to the pilot's peripheral and forward vision.

Referring also to FIGS. 2A and 2B , the primary avionics display 102 may display a full-screen background interface 110 as well as one or more multifunction windows (MFW) 112 in which foreground interfaces are displayed. Each foreground interface may be displayed in its own MFW 112 , entirely within the boundaries of the background interface 110 , whereby a portion of the background interface 110 can be seen between the edges of each MFW 112 and the edges of the primary avionics display 102 . For example, the pilot-side primary avionics display 102 a may have a default layout as shown by FIG. 2A , displaying a synthetic vision system (SVS) 110 a as its background interface 110 . The SVS 110 a may include a flight mode annunciator (FMA) display 110 b and a horizontal situation indicator (HSI) display and primary flight indicators 110 c . The primary avionics display 102 a may display foreground interfaces in quarter-size MFWs 112 a , 112 b and half-size MFW 112 c . The MFWs 112 displayed by the primary avionics display 102 may include any combination of map displays, synoptic displays, engine indicator and crew alert system (EICAS) displays, chart displays (e.g., approach charts), systems management, and video displays. Similarly, the co-pilot-side primary avionics display 102 b as shown by FIG. 2B may have a default layout including a full screen map display 110 d displayed as its background interface 110 . Foreground interfaces may be displayed by the primary avionics display 102 b in quarter-sized MFWs 112 d and 112 e , and a Primary Flight Display (PFD) may be displayed in half-size MFW 112 f.

Referring to FIGS. 3A and 3B , the primary avionics displays 102 may provide simplified format control via finger contacts or gestures performed by a pilot, co-pilot, or crew member. For example, FIG. 3A illustrates the pilot-side primary avionics display 102 a . A pilot or other crewmember may generate a quarter-size format selection menu (FSM) 114 a associated with the foreground interface (ex.—application) displayed in the corresponding quarter-size MFW 112 a by touching and holding one or more fingers ( 116 ) to the quadrant of the surface of the pilot-side primary avionics display 102 a corresponding to the MFW 112 a in which the foreground interface is displayed. For example, “touching or holding” one or more fingers to a display (including any primary avionics display 102 , secondary avionics display 104 , format selector panel 106 or keyboard panel 108 ) refers to pressing the one or more fingers to the surface of a display and holding the one of more fingers in place (until the avionics display system 100 generates a response). A “touch-and-hold” may be followed by sliding the one or more fingers across the display surface as explained below. A similar FSM (not shown) corresponding to the interface displayed in the MFW 112 b may be generated by touching and holding the one or more fingers to the MFW 112 b in the lower left quadrant of the screen. Referring to FIG. 3B , the pilot may generate a half-size format selection menu 114 b over the background interface 110 a (i.e., an area of the background interface 110 over which a MFW 112 is not currently displayed) by touching and holding at least one finger ( 116 ) to a center-left portion of the surface of the pilot-side primary avionics display 102 a , between the upper left and lower left display quadrants. A similar half-size FSM 114 b (not shown) may be generated by a touch-and-hold performed in the center-right portion of the surface of the primary avionics display 102 . Format selection menus 114 associated with foreground interfaces displayed in MFWs 112 by the secondary avionics display 104 may be generated in a similar fashion.

Referring to FIGS. 3C and 3D , the size or position of a MFW 112 may be changed by finger contacts or gestures. Referring to FIG. 3C , a pilot or crewmember may touch and hold one or more fingers to a swipe bar 118 a placed at the bottom edge of the MFW 112 a and swipe the finger directly downward ( 120 a ) along the surface of the primary avionics display 102 a . The MFW 112 a may expand downward in size from a quarter-size MFW to a half-size MFW, as shown by the lower left figure of MFW 112 a . The presence of a swipe bar 118 along any edge of a MFW 112 may indicate the ability to expand or retract the MFW 112 by a touch-and-hold followed by a directional swipe 120 . For example, the half-size MFW 112 c includes swipe bars 118 b and 118 c along its top and bottom edges respectively. The pilot may touch and hold one or more fingers to the top swipe bar 118 b and swipe directly downward ( 120 b ), reducing the size of the MFW 112 c from half-size to quarter-size, as shown by the upper right figure of the MFW 112 c . Referring to FIG. 3D , the pilot may touch and hold one or more fingers within the edges of a MFW 112 a and swipe horizontally, vertically, or diagonally ( 122 ) toward a new quadrant (e.g., the lower right quadrant) of the pilot-side primary avionics display 102 a . The primary avionics display 102 a may guide the swipe by displaying a silhouette ( 112 g ) of the MFW 112 a until the destination quadrant (or a silhouette of the growing/shrinking MFW, for a resizing swipe) is reached. Once the swipe has reached the destination quadrant, the MFW 112 a may be repositioned from the original (upper left) quadrant to the destination (lower right) quadrant of the pilot-side primary avionics display 102 a . Foreground interfaces displayed in MFWs 112 by the secondary avionics display 104 may be repositioned in a similar fashion.

Referring to FIGS. 4A and 4B , the format selector panels (FSP) 106 similarly provide a pilot or crewmember with access to format selector menus (FSM) 114 (the FSMs corresponding to applications or interfaces displayed in MFWs 112 ) via finger contacts and gestures. For example, as shown by FIG. 4A , the pilot-side FSP 106 a may include a series of selectors 122 tying the FSP 106 a to the pilot-side primary avionics display 102 a . Selectors 122 a and 122 b , along the left edge of the FSP 106 a , may provide FSMs 114 a corresponding to the MFWs 112 a and 112 b positioned along the left side of the pilot-side primary avionics display 102 a (as shown by FIG. 2A ). By pressing and holding two fingers ( 116 ) to the selector 122 a , the FSP 106 a may display a format selector menu 114 a corresponding to the MFW 112 a displayed by the primary avionics display 102 a . A FSP 106 may be configured to control (ex.—tied to) either primary avionics display ( 102 a , 102 b ) or the secondary avionics display 104 a . For example, the indicator 124 a of the pilot-side FSP 106 a may show that the pilot-side FSP 106 a is currently controlling the pilot-side primary avionics display 102 a , while the indicators 124 b and 124 c correspond to the co-pilot-side primary avionics display 102 a and the secondary avionics display 104 respectively. By pressing either indicator 124 b or 124 c , a pilot or crewmember may access a series of selectors 122 (not shown) tied to the background interfaces 110 , foreground interfaces, and MFWs 112 displayed by the co-pilot-side primary avionics display 102 b or the secondary avionics display 104 . The pilot-side FSP 106 a may include additional selectors 122 c (corresponding to MFW 112 c , on the right side of the pilot-side primary avionics display 102 a ), 122 d (currently unused), and 122 e (corresponding to the SVS display 110 a displayed by the full-screen background interface 110 of the pilot-side primary avionics display 102 a ). As with a MFW 112 displayed by a primary avionics display 102 or a secondary avionics display 104 , a selector 122 displayed by an FSP 106 may be resized by a single finger swipe 120 . For example, swipe bars 118 a and 118 b provide for horizontal and vertical resizing of the selector 122 a , while swipe bars 118 c and 118 d provide for vertical resizing of selectors 122 b and 122 c respectively. As shown by FIG. 4B , a pilot may activate a particular selector 122 by tapping one or more fingers ( 116 ) to the corresponding MFW 112 of the primary avionics display 102 . Tapping (as opposed to “touching and holding” refers to making finger contact with the display surface and promptly removing the one or more fingers once a single, brief contact with the display surface is made. For example, the pilot may tap ( 116 ) on a MFW 112 a displayed by the pilot-side primary avionics display 102 a , thereby highlighting the corresponding selector 122 a on the pilot-side FSP 106 a (which controls the pilot-side primary avionics display 102 a , as shown by the top-left indicator 124 a ). A selector 122 associated with an interface displayed in a MFW 112 by the secondary avionics display 104 may be displayed by the FSP 106 in the same fashion, i.e., by tapping the corresponding MFW (not shown) displayed by the secondary avionics display 104 .

Referring to FIG. 5 , the secondary avionics display 104 may display the same selection of background interfaces 110 or foreground interfaces displayed in MFWs 112 as displayed by either primary avionics display 102 . Similarly, as shown by FIGS. 4A and 4B , a pilot may access format selector menus 114 corresponding to, and controlling, the interfaces displayed by the secondary avionics display 104 via either format selector panel 106 by selecting the appropriate indicator 124 c (as shown by FIG. 4A ). The secondary avionics display 104 may also be used as a rehosting display for one or more pads, tablets, or similar electronic flight bag (EFB) devices 126 . Instead of requiring the pilot to position the EFB device 126 where it can be accessed directly, the secondary avionics display 104 may link ( 128 ) to the EFB device 126 . For example, the link 128 may include either a wireless link 128 a or a physical (ex.—wired) link 128 b . The EFB device 126 may be activated and stored securely in the cockpit (e.g., in a flight bag). The secondary avionics display 104 may generate a mirror interface ( 130 a , 130 b ) which mirrors the display windows of EFB devices 126 a and 126 b (each EFB device 126 corresponding to a pilot, co-pilot, or crewmember), providing the pilot and crew with access to any EFB applications or windows running on the EFB devices 126 a , 126 b . Any required control interface interaction with the EFB devices 126 a , 126 b may be provided by finger contacts or gestures with the mirror interfaces 130 a , 130 b of the secondary avionics display 104 , which relays any related commands and avionics events to the EFB devices 126 a , 126 b via the wireless links 128 a , 128 b . The pilot may toggle the output of the secondary avionics display 104 between avionics interfaces (as displayed by the primary avionics display 102 ) and mirror interfaces 130 (as shown by FIG. 5 ) via either format selector panel 106 (as shown by FIG. 4A ), wherein the indicator 124 c indicates that the format selector panel 106 is currently tied to the secondary avionics display 104 . The secondary avionics display 104 may further incorporate one or more handrests for use by a pilot or co-pilot during turbulent air conditions.

Referring to FIGS. 6A and 6B , the keyboard panels 108 of the avionics flight display 100 may generate a default interface 132 which, in conjunction with the primary avionics display 102 , provides radio tuning, audio, and communications functionality. For example, referring to FIG. 6A , an adjacent portion of the primary avionics display 102 may display active and standby information corresponding to communications channels 134 a and 134 b , navigation channels 134 c and 134 d , and air traffic control (ATC) channel 134 e . The corresponding keyboard panel 108 may provide touch-selectable indicators 136 corresponding to the current status of each channel 134 (e.g., actively monitoring, deep control, microphone on). Selectable indicators 138 a , 138 b may be tapped to activate or deactivate a microphone associated with communications channels 134 a , 134 b . The default interface 132 of the keyboard panel 108 may include a numeric keypad 140 for resetting communications or transponder frequencies associated with communications channels 134 a - 134 e.

Referring to FIG. 6B , the keyboard panel 108 may display an alternative interface 142 when alphanumeric input is required by the avionics display system 100 . For example, the pilot may tap ( 144 ) on any text field 146 of a primary avionics display 102 or secondary avionics display (not shown), thereby inducing ( 148 a ) the corresponding keyboard panel 108 to display its alternative interface 142 . The alternative interface 142 may include a full touch-sensitive alphanumeric keyboard, allowing the pilot or crewmember to enter any necessary information into the highlighted text field 146 . The alternative interface 142 may be timed, such that when the keyboard panel 108 activates its alternative interface 142 , and a predetermined time threshold is reached (e.g., 20 seconds) during which no finger contact is made with the alphanumeric keyboard of the alternative interface 142 , the keyboard panel 108 may revert ( 148 b ) to its default interface 132 , once again displaying interactive radio control information as shown by FIG. 6A .

Referring back to FIG. 1 , the avionics flight display 100 may include one or more physical buttons 152 and one or more volume sliders 154 . For example, the physical buttons 152 may include a pilot button, may be used to switch the pilot-side and co-pilot-side display components. For example, the co-pilot-side primary avionics display 102 b would generate a SVS 110 a in the background (rather than a map display 110 d ), while the pilot-side primary avionics display 102 a would generate a map display 110 d in the background (rather than a SVS 110 a ). Foreground interfaces displayed in MFWs 112 may likewise switch from pilot-side primary avionics display 102 a to co-pilot-side primary avionics display 102 b , and vice versa. Pushing the button 152 a second time may reset the pilot-side and co-pilot-side display components to their prior settings. The physical buttons 152 may include a home button, which when pushed causes the format selector panel 106 to display a format selector menu 114 a (as shown by FIG. 4A ). The volume sliders 154 may be used to increase or reduce the operating volume associated with a radio or communications channel 134 controlled by the default interface 132 displayed by the keyboard panel 108 (as shown by FIG. 6A ). For example, tapping at least one finger on a portion of the keyboard panel 108 corresponding to a radio frequency or channel may tie the volume slider 154 (as shown by FIG. 1 ) corresponding to the keyboard panel 108 to that particular radio frequency or channel. The pilot may adjust the volume of the radio frequency or channel by sliding the corresponding volume slider 154 up or down.

FIG. 7 illustrates a method 200 for format control via an avionics display system 100 according to embodiments of the inventive concepts disclosed herein. At a step 210 , a pilot generates a response via a primary avionics display 102 of the avionics display system 100 by at least one of: 1) touching and holding at least one finger to the primary avionics display 102 , or to a first interface displayed by the primary avionics display 102 ; or 2) tapping the at least one finger to the primary avionics display 102 or on the first interface. The first interface may include a foreground interface displayed in a MFW 112 or a background interface 110 .

For example, the pilot may generate a quarter-size menu 114 associated with an MFW 112 displayed in a quadrant of the primary avionics display 102 by touching and holding (or tapping) the at least one finger to ( 116 ) the MFW 112 . The pilot may generate a half-size menu associated with the primary avionics display 102 in either a center-left or center-right portion of the primary avionics display by touching and holding (or tapping) the at least one finger to ( 116 ) the center-left portion or the center-right portion of the primary avionics display. The pilot may resize the MFW 112 by touching and holding the at least one finger to an edge 118 of the MFW 112 and sliding the at least one finger ( 120 ) along the primary avionics display 102 in a vertical or a horizontal direction. The pilot may reposition the MFW 112 from a first quadrant of the primary avionics display 102 to a second quadrant by touching and holding the at least one finger to the MFW 112 and sliding the at least one finger ( 122 ) along the primary avionics display toward the second display quadrant.

At a step 220 , the pilot generates a response via a secondary avionics display 104 of the avionics display system 100 by at least one of: 1) touching and holding at least one finger to the secondary avionics display 104 , or to a second interface displayed by the primary avionics display 104 ; or 2) tapping the at least one finger to the secondary avionics display 104 or on the second interface. The second interface may include a foreground interface displayed in a MFW 112 , a background interface 110 , or a mirror interface 130 .

For example, the pilot may generate a quarter-size menu 114 associated with an MFW 112 displayed in a quadrant of the secondary avionics display 104 by touching and holding (or tapping) the at least one finger to ( 116 ) the MFW 112 . The pilot may generate a half-size menu associated with the secondary avionics display 104 in either a center-left or center-right portion of the secondary avionics display by touching and holding (or tapping) the at least one finger to ( 116 ) the center-left portion or the center-right portion of the secondary avionics display. The pilot may resize the MFW 112 by touching and holding the at least one finger to an edge 118 of the MFW 112 and sliding the at least one finger ( 120 ) along the secondary avionics display 104 in a vertical or a horizontal direction. The pilot may reposition the MFW 112 from a first quadrant of the secondary avionics display 104 to a second quadrant by touching and holding the at least one finger to the MFW 112 and sliding the at least one finger ( 122 ) along the secondary avionics display toward the second display quadrant.

The method 200 may include additional steps 230 , 240 , and 250 . At a step 230 , the pilot generates, via a format selector panel 106 of the avionics display system 100 , a menu associated with the primary avionics display 102 or a first interface displayed by the primary avionics display by touching and holding (or tapping) the at least one finger to ( 116 ) the first interface. At a step 240 , the pilot generates, via the format selector panel 106 , a menu associated with the secondary avionics display 104 or a second interface displayed by the secondary avionics display by touching and holding (or tapping) the at least one finger to ( 116 ) the second interface. At a step 250 , the pilot generates, via a keyboard panel 108 of the avionics display system 100 , an interface 142 configured to accept alphanumeric input from a user by tapping at least one finger ( 144 ) on a text field 146 displayed by the primary avionics display 102 or the secondary avionics display 104 .

While particular aspects of the inventive concepts disclosed herein have been shown and described, it will be apparent to those skilled in the art that, based upon the teachings herein, changes and modifications may be made without departing from the scope of the inventive concepts disclosed herein and their broader aspects and, therefore, the appended claims are to encompass within their scope all such changes and modifications as are within the broad scope of the inventive concepts disclosed herein.