Infrared light suppression in eye-tracking imaging

A head mounted device is a wearable optical system, typically worn on the head of a user. Head mounted devices may include one or more electronic components for use in a variety of applications, such as gaming, aviation, engineering, medicine, entertainment, activity tracking, and so on. Some head mounted devices may perform eye-tracking which may enhance the user's viewing experience. Eye-tracking may be aided, in some cases, by illuminating the eye of the user. Thus, some conventional head mounted devices may incorporate an eye-tracking system that includes an illumination source as well as a camera for tracking movements of the user's eye. However, various contexts may generate challenges to capturing images of the eye that include sufficient contrast for analyzing the images. In particular, stray light may propagate through the optical system to the camera, thereby increasing noise in the captured images.

Non-limiting and non-exhaustive embodiments of the invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.

FIG. 1 illustrates an example head mounted display (HMD), in accordance with aspects of the present disclosure.

FIG. 2 is a top view of an example near-eye optical element that includes a transparent layer, an illumination layer, an optical combiner layer, and a display layer.

FIG. 3 illustrates a front view of an eye through an example illumination layer, in accordance with aspects of the disclosure.

FIG. 4 illustrates an example optical system including an infrared absorber layer, in accordance with aspects of the disclosure.

Embodiments of an optical system, a near-eye optical element, and a head-mounted device for infrared light suppression in eye-tracking imaging are described herein. In the following description, numerous specific details are set forth to provide a thorough understanding of the embodiments. One skilled in the relevant art will recognize, however, that the techniques described herein can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

The system and device for suppressing infrared light in eye-tracking images that are described in this disclosure include incorporating infrared absorbing components into an optical system. Optical systems that perform eye-tracking may include one or more infrared light sources to illuminate the eye with infrared light as well as an eye-tracking camera to image the eye in response to reflected infrared light that reflects off the eye. In some systems, interference infrared light may become incident on the camera, thereby reducing the effective signal-to-noise ratio of the eye-tracking system. The interference infrared light may be caused by infrared illumination light unintentionally reflecting off one or more surfaces/interfaces of the optical system. Some optical systems may include one or more multi-layer anti-reflective (AR) coatings applied to one or more surfaces to allow the interference infrared light to escape the optical system rather than being reflected back towards the camera. Other infrared interference light may also be caused by external infrared light being incident on the optical system, such as infrared light received from the environment. Thus, some optical systems may include an infrared reflecting layer to prevent external infrared light from propagating through the system.

In contrast to the AR coatings and infrared reflecting layers discussed above, embodiments of this disclosure include an infrared absorber layer that is strategically positioned within the optical system (i.e., within the stack of optical elements) to suppress/absorb infrared interference light. The infrared absorber layers in this disclosure may be specially tuned to transmit a very-high percentage of visible light while absorbing infrared interference light. Optical systems of the disclosure may benefit from the transmission of visible light and the suppression of infrared interference light due to the requirements of an optical system to pass visible scene light and/or visible display light, in the context of a head mounted device. These and other embodiments are described in more detail in connection with FIGS. 1-4 .

FIG. 1 illustrates an example HMD 100 , in accordance with aspects of the present disclosure. The illustrated example of HMD 100 is shown as including a frame 102 , temple arms 104 A and 104 B, and near-eye optical elements 110 A and 110 B. Eye-tracking cameras 108 A and 108 B are shown as coupled to temple arms 104 A and 104 B, respectively. FIG. 1 also illustrates an exploded view of an example of near-eye optical element 110 A. Near-eye optical element 110 A is shown as including an optically transparent layer 120 A, an illumination layer 130 A, an optical combiner layer 140 A, and a display layer 150 A. Illumination layer 130 A is shown as including a plurality of in-field infrared light sources 126 . The in-field infrared light source 126 may be configured to emit infrared illumination light for eye-tracking purposes, for example. Display layer 150 A may include a waveguide 158 A that is configured to direct virtual images to an eye of a user of HMD 100 .

An HMD, such as HMD 100 is one type of head mounted device, typically worn on the head of a user to provide artificial reality content to a user. Artificial reality is a form of reality that has been adjusted in some manner before presentation to the user, which may include, e.g., virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and/or derivative thereof.

As shown in FIG. 1 , frame 102 is coupled to temple arms 104 A and 104 B for securing the HMD 100 to the head of a user. Example HMD 100 may also include supporting hardware incorporated into the frame 102 and/or temple arms 104 A and 104 B. The hardware of HMD 100 may include any of processing logic, wired and/or wireless data interfaces for sending and receiving data, graphic processors, and one or more memories for storing data and computer-executable instructions. In one example, HMD 100 may be configured to receive wired power and/or may be configured to be powered by one or more batteries. In addition, HMD 100 may be configured to receive wired and/or wireless data including video data.

FIG. 1 illustrates near-eye optical elements 110 A and 110 B that are configured to be mounted to the frame 102 . In some examples, near-eye optical elements 110 A and 110 B may appear transparent to the user to facilitate augmented reality or mixed reality such that the user can view visible scene light 191 from the environment while also receiving display light directed to their eye(s) by way of display layer 150 A. In further examples, some or all of the near-eye optical elements 110 A and 110 B may be incorporated into a virtual reality headset where the transparent nature of the near-eye optical elements 110 A and 110 B allows the user to view an electronic display (e.g., a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a micro-LED display, etc.) incorporated in the virtual reality headset.

As shown in FIG. 1 , illumination layer 130 A includes a plurality of in-field infrared light sources 126 . Each in-field infrared light source 126 may be disposed on a transparent substrate and may be configured to emit infrared illumination light towards an eyeward side 109 of the near-eye optical element 110 A. In some aspects of the disclosure, the in-field infrared light sources 126 are configured to emit near infrared light (e.g. 700 nm-1.4 μm). Each in-field infrared light source 126 may be a micro light emitting diode (micro-LED), an edge emitting LED, a vertical cavity surface emitting laser (VCSEL) diode, or a Superluminescent diode (SLED). In other embodiments, each in-field infrared light source 126 may be an exit feature of a waveguide, included in illumination layer 130 A, where the waveguide is configured to direct infrared light emitted from a VCSEL diode or other light source incorporated into the rim of frame 102 to the exit feature for illuminating the eye of the user.

Conventional eye-tracking solutions may provide light sources disposed around a rim/periphery of a lens. However, placing light sources within the field of view of the eye may be advantageous for computation of specular or “glint” reflections that can be imaged by a camera such as eye-tracking camera 108 A that is positioned to image the eye of a wearer of HMD 100 .

While in-field infrared light sources 126 may introduce minor occlusions into the near-eye optical element 110 A, the in-field infrared light sources 126 , as well as their corresponding routing may be so small as to be unnoticeable or insignificant to a wearer of HMD 100 . Additionally, any occlusion from in-field infrared light sources 126 will be placed so close to the eye as to be unfocusable by the human eye and therefore assist in the in-field infrared light sources 126 being not noticeable or insignificant. In some embodiments, each in-field infrared light source 126 has a footprint (or size) that is less than about 200×200 microns.

As mentioned above, the in-field infrared light sources 126 of the illumination layer 130 A may be configured to emit infrared illumination light towards the eyeward side 109 of the near-eye optical element 110 A to illuminate the eye of a user. The near-eye optical element 110 A is shown as including optical combiner layer 140 A disposed between the illumination layer 130 A and a backside 111 of the near-eye optical element 110 A. In some aspects, the optical combiner layer 140 A is configured to receive reflected infrared light that is reflected by the eye of the user and to direct the reflected infrared light towards the eye-tracking camera 108 A. In some examples, the eye-tracking camera 108 A is an infrared camera configured to image the eye of the user based on the received reflected infrared light. In some aspects, the optical combiner layer 140 A is transmissive to visible light, such as scene light 191 incident on the backside 111 of the near-eye optical element 110 A. In some examples, the optical combiner layer 140 A may be configured as a volume hologram and/or may include one or more Bragg gratings for directing the reflected infrared light towards the eye-tracking camera 108 A. In some examples, the optical combiner layer 140 A includes a polarization-selective hologram (a.k.a. polarized volume hologram) that diffracts a particular polarization orientation of incident light while passing other polarization orientations. In other examples, the optical combiner layer 140 A includes one or more Fresnel optical elements that are configured to direct the reflected infrared light to the camera while also allowing visible light to propagate through the near-eye optical element 110 A. By way of example a Fresnel optical element, included in the optical combiner layer 140 A, may include active surfaces that are selectively coated with a “hot mirror” layer (reflecting infrared light and passing visible light) to direct infrared light to a camera while also passing visible light for viewing by the wearer of the HMD 100 .

Display layer 150 A may include one or more other optical elements depending on the design of the HMD 100 . For example, the display layer 150 A may include a waveguide 158 A to direct display light generated by an electronic display to the eye of the user. In some implementations, at least a portion of the electronic display is included in the frame 102 of the HMD 100 . The electronic display may include an LCD, an organic light emitting diode (OLED) display, micro-LED display, pico-projector, or liquid crystal on silicon (LCOS) display for generating the display light.

Optically transparent layer 120 A is shown as being disposed between the illumination layer 130 A and the eyeward side 109 of the near-eye optical element 110 A. The optically transparent layer 120 A may receive the infrared illumination light emitted by the illumination layer 130 A and pass the infrared illumination light to illuminate the eye of the user. As mentioned above, the optically transparent layer 120 A may also be transparent to visible light, such as scene light 191 received from the environment and/or display light received from the display layer 150 A. In some examples, the optically transparent layer 120 A has a curvature for focusing light (e.g., display light and/or scene light) to the eye of the user. Thus, the optically transparent layer 120 A may, in some examples, may be referred to as a lens. In some aspects, the optically transparent layer 120 A has a thickness and/or curvature that corresponds to the specifications of a user. In other words, the optically transparent layer 120 A may be a prescription lens. However, in other examples, the optically transparent layer 120 A may be a non-prescription lens.

FIG. 2 is a top view of an example near-eye optical element 210 that includes a transparent layer 220 , an illumination layer 230 , an optical combiner layer 240 , and a display layer 250 . A plurality of infrared light sources 237 A- 237 E emit infrared illumination light 239 to an eyebox area 207 on the eyeward side 109 of the near-eye optical element 210 to illuminate eye 206 . The different infrared light sources 237 A- 237 E may direct infrared illumination light 239 to eye 206 at different angles depending on the position of the infrared light sources 237 A- 237 E with respect to eye 206 . For example, infrared light sources 237 A and 237 E may include beam-forming elements that direct the infrared illumination light to eye 206 at steeper angles compared to infrared light source 237 C directing infrared illumination light 239 to eye 206 at an angle closer to normal. As described above, infrared light sources 237 A- 237 E may be VCSELs or SLEDs, and consequently infrared illumination light 239 may be narrow-band infrared illumination light (e.g. linewidth of 1-10 nm).

Eye 206 reflects at least a portion of the infrared illumination light 239 back to near-eye optical element 210 as reflected infrared light (not illustrated in FIG. 2 ) and the reflected infrared light propagates through layers 220 and 230 before encountering the optical combiner layer 240 . Optical combiner layer 240 is configured to receive the reflected infrared light and direct the reflected infrared light to the camera 108 A to generate eye-tracking images. As an example, FIG. 4 illustrates reflected infrared light propagating along optical path 460 ( 2 ) being redirected to camera 108 A by optical combiner layer 440 .

Returning to FIG. 2 , camera 108 A is configured to capture eye-tracking images of eye 206 . In some examples, camera 108 A includes an infrared bandpass filter having a center wavelength corresponding to an infrared wavelength of the infrared illumination light 239 . Thus, the bandpass filter may pass the wavelength of the infrared illumination light 239 emitted by the infrared light sources and block other light from becoming incident on an image sensor of camera 108 A. Camera 108 A may include a complementary metal-oxide semiconductor (CMOS) image sensor.

FIG. 2 shows that scene light 191 (visible and infrared light) from the external environment may propagate through display layer 250 , optical combiner layer 240 , illumination layer 230 , and transparent layer 220 to become incident on eye 206 so that a user can view the scene of an external environment. FIG. 2 shows that display layer 250 may generate or redirect display light 293 to present virtual images to eye 206 . Display light 293 is visible light and propagates through the optical combiner layer 240 , illumination layer 230 , and transparent layer 220 to reach eye 206 .

In some examples, transparent layer 220 may include a base curvature 221 that is the surface closest to eyeward side 109 . Base curvature 221 may be configured to focus a virtual image included in display light 293 for an eye of a user. Base curvature 221 may be formed in a refractive material 222 of transparent layer 220 using a subtractive process. The refractive material 222 may have a refractive index of approximately 1.5, in some embodiments. Illumination layer 230 may include a transparent material 232 that may encapsulate the infrared illuminators 237 . Transparent material 232 and refractive material 222 are configured to transmit visible light (e.g. 400 nm-700 nm) and near-infrared light (e.g. 700 nm-1.4 μm).

FIG. 3 illustrates a front view of eye 206 through an example illumination layer 330 , in accordance with aspects of the disclosure. In the illustrated embodiment, illumination layer 330 include twenty-one infrared light sources ( 337 A- 337 U). In the illustrated example, infrared light sources 337 A- 337 H may be considered an “inner ring” of infrared light sources while infrared light sources 337 I- 337 U are considered an “outer ring” of infrared light sources. In some examples, infrared light sources 337 I- 337 U may direct their infrared illumination light to eye 206 at a steeper angle (e.g. 14-25 degrees) than infrared light sources 337 A- 337 H in the inner ring (e.g. 0-22 degrees).

FIG. 4 illustrates an example optical system 400 including an infrared absorber layer 425 , in accordance with aspects of the disclosure. The illustrated example of optical system 400 is shown as including camera 108 A, a transparent layer 420 , an illumination layer 430 , an optical combiner layer 440 , an optional anti-reflective (AR) coating 423 , infrared absorber layer 425 , and an additional layer 450 . The illumination layer 430 is shown as including several infrared light sources, such as infrared light source 237 A.

As shown in FIG. 4 , infrared light source 237 A is configured to generate and then direct narrow-band infrared illumination light to an eyebox area 207 along optical path 460 ( 1 ). FIG. 4 further illustrates the narrow-band infrared illumination light reflecting off the eye 206 and then returning to the optical system along optical path 460 ( 2 ). The reflected infrared light then passes through the optional AR coating 423 , through the transparent layer 420 , through the illumination layer 430 , and then is incident on the optical combiner layer 440 .

As shown, the optical combiner layer 440 then directs the reflected infrared light to the camera 108 A on the eyeward side 109 along optical path 460 ( 3 ). However, some reflected infrared light (e.g., reflected infrared light received at the optical combiner layer 440 at high angles of incidence) may propagate through the optical combiner layer 440 rather than being directed back towards the camera 108 A. In the absence of infrared absorber layer 425 , this reflected infrared light that passes through the optical combiner layer 440 may reflect off one or more other layers or surfaces of the optical system, such as additional layer 450 . For example, after passing through the optical combiner layer 440 , the reflected infrared light may reflect off the additional layer 450 (e.g., display layer 250 of FIG. 2 and/or a backside transparent layer or lens). In some configurations, there may be a physical air gap between one or more of the layers, such as between the optical combiner layer 440 and the additional layer 450 and/or between one or more components of the additional layer 450 , itself. These air gaps may increase the likelihood of unwanted infrared light being reflected back towards the camera 108 A. Regardless, the stray infrared light may then be received by the camera 108 A as interference infrared light, appearing as noise in the resultant eye-tracking images. In addition, scene light 191 may include external infrared light that is received from the environment. This external infrared light may be received by the camera 108 A, also as interference infrared light. The interference infrared light, whether it be reflected infrared light that propagates through the optical combiner layer 440 or external infrared light received from the backside 111 of the optical system 400 may appear as noise in the images captured by camera 108 A, effectively reducing the signal-to-noise ratio of the eye-tracking system. Accordingly, embodiments of the present disclosure include incorporating an infrared absorbing layer 425 on the backside of the optical combiner layer 440 to suppress such infrared interference light.

In some aspects, the infrared absorbing layer 425 is a paint (e.g., a wet coat) that is applied to a surface (i.e., on the backside 111 ) of the optical combiner layer 440 . The paint may include a homogenous solution of an infrared absorbing solute or dye dissolved in a solvent, such as methanol or acetone. In other examples, infrared absorbing layer 425 may be a thin film (e.g., a laminate) that is adhered to the backside 111 of the optical combiner layer 440 . In yet another aspect, the infrared absorbing layer 425 may be fabricated in situ as an internal layer of the optical combiner layer 440 . As shown, the infrared absorbing layer 425 may be disposed between the optical combiner layer 440 and the other layer 450 (e.g., display layer and/or backside lens). In some examples, infrared absorbing layer 425 is disposed directly on the surface of the optical combiner layer 440 (i.e., without any other intervening optical interfaces therebetween).

As mentioned above, infrared absorbing layer 425 may be specially tuned to transmit a high percentage of visible light while absorbing infrared interference light. By way of example, the infrared absorbing layer 425 may be configured to absorb 70% or more of the infrared interference light that is incident on the infrared absorbing layer 425 , while allowing 90% or more of visible light to pass through the infrared absorbing layer 425 . In some embodiments, the absorptivity/transmissiveness of the infrared absorbing layer scales across both the visible spectrum and the infrared spectrum, such that a lower absorptivity in the infrared spectrum means that the impact in the visible spectrum may be proportionally reduced. Accordingly, in some aspects, the absorptivity of the infrared absorbing layer 425 need not be maximized to provide sufficient absorption of the infrared interference light while also providing sufficient transmission of visible light. Test results performed by applicant nonintuitively indicate that a lower absorptivity (e.g., 20% absorptivity to infrared interference light) may provide adequate absorption of the infrared interference light while simultaneously providing very little absorption in the visible spectrum (e.g., 1% or less). In some examples, the infrared absorptivity of the infrared absorbing layer 425 may be tuned by adjusting the thickness of infrared absorbing layer 425 and/or adjusting the concentration of infrared absorbing material including the infrared absorbing layer 425 .

As mentioned above, the infrared illumination light emitted by the infrared light source 237 A may be narrow-band infrared illumination light (e.g. linewidth of 1-10 nm). In some examples, the narrow-band infrared illumination light is centered around an infrared wavelength (e.g., 940 nm). In addition, the camera 108 A may include an infrared bandpass filter having a center wavelength corresponding to the wavelength of the narrow-band infrared illumination light. For example, the bandpass filter may have a center wavelength that is the same as (e.g., 940 nm), or at least overlaps with the narrow-band of the infrared illumination light. Accordingly, the infrared absorber layer 425 may be tuned to have a peak absorptivity at a wavelength that corresponds to that of the narrow-band infrared illumination light (e.g., infrared absorber layer 425 may have a peak absorptivity at 940 nm).

In some examples, the infrared absorber layer 425 may be tuned for very-high transmission for visible light and for very-high absorption over a significant range of narrow-band infrared light (937 nm-950 nm). The infrared absorber layer 425 may also be specifically tuned for very-high absorption of narrow-band infrared illumination light corresponding with the infrared light source 237 A emitting 940 nm light, for example. In other embodiments, the narrow-band of infrared light may have different wavelengths. The transmission spectrum of an infrared absorber layer may be tuned for less than very-high absorption of at least a portion of gap-light having wavelengths between the visible light and the narrow-band of infrared light (e.g. 700 nm-937 nm).

Embodiments of the invention may include or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that has been adjusted in some manner before presentation to a user, which may include, e.g., a virtual reality (VR), an augmented reality (AR), a mixed reality (MR), a hybrid reality, or some combination and/or derivatives thereof. Artificial reality content may include completely generated content or generated content combined with captured (e.g., real-world) content. The artificial reality content may include video, audio, haptic feedback, or some combination thereof, and any of which may be presented in a single channel or in multiple channels (such as stereo video that produces a three-dimensional effect to the viewer). Additionally, in some embodiments, artificial reality may also be associated with applications, products, accessories, services, or some combination thereof, that are used to, e.g., create content in an artificial reality and/or are otherwise used in (e.g., perform activities in) an artificial reality. The artificial reality system that provides the artificial reality content may be implemented on various platforms, including a head-mounted display (HMD) connected to a host computer system, a standalone HMD, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

The above description of illustrated embodiments of the invention, including what is described in the Abstract, is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize.

These modifications can be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification. Rather, the scope of the invention is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation.