Projector alignment in a contact lens

1. Technical Field

This disclosure relates generally to projectors for electronic contact lenses.

2. Description of Related Art

Augmented reality (AR) adds computer-generated information to a person's view of the world around them. One type of AR system includes an electronic contact lens with tiny video projectors (or “femtoprojectors”) as described in Deering (U.S. Pat. No. 8,786,675). Multiple projectors can be included within an electronic contact lens, and can each project different portions of the same AR interface. Aligning the projectors within a contact lens can ensure that the AR interface is displayed to a wearer of the contact lens properly.

Embodiments of the disclosure have other advantages and features which will be more readily apparent from the following detailed description and the appended claims, when taken in conjunction with the examples in the accompanying drawings, in which:

FIG. 1A shows a user wearing an eye-mounted display in communication with an auxiliary necklace.

FIG. 1B shows a magnified view of the eye-mounted display mounted on the user's eye.

FIG. 2A shows a cross sectional view of an eye-mounted display containing multiple femtoprojectors in a contact lens.

FIG. 2B shows a front view of an eye-mounted display containing multiple femtoprojectors in a contact lens.

FIG. 2C shows images projected onto the retina by the femtoprojectors of FIGS. 2A and 2B .

FIG. 3A shows a set of misaligned test images projected onto the retina by a set of femtoprojectors in a contact lens.

FIG. 3B shows a set of aligned test images projected onto the retina by a set of femtoprojectors in a contact lens.

FIG. 4A shows a rotationally misaligned test image in a set of laterally misaligned test images projected onto the retina by a set of femtoprojectors in a contact lens.

FIG. 4B shows an image formed from tiled sub-images with blank spaces between the sub-images projected onto the retina by a set of femtoprojectors in a contact lens.

FIG. 4C shows a set of aligned test images projected onto the retina by a set of femtoprojectors.

The figures and the following description relate to preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of what is claimed.

Introduction

A contact lens can include multiple femtoprojectors within the contact lens. Each femtoprojector includes an image source and an optical system that projects images from the image source onto a user's retina. Different femtoprojectors project to different areas of the retina. In one approach, some femtoprojector(s) project images onto the fovea and other femtoprojectors project images onto peripheral regions of the retina. If the images are overlapping, they can be stitched together to create a larger image in the aggregate. For both overlapping and non-overlapping images, alignment of the images is important for proper display.

Images projected by femtoprojectors can be aligned by an alignment circuit within the contact lens. The alignment circuit can adjust the physical position or orientation of one or more femtoprojectors within the contact lens. For example, the alignment circuit can include an electroactive polymer that, when activated, can move a femtoprojector relative to the contact lens. In addition, the alignment circuit can configure a location within a femtoprojector array from which an image is projected. For example, the alignment circuit can provide a reference position within a femtoprojector array, and a femtoprojector can project an image from a position relative to the reference position.

A user can provide an input to the alignment circuit in order to align femtoprojectors. For instance, the user can provide input using one or more of a smart device user interface, a physical input device (e.g., a knob), a voice command, a hand gesture, and an eye gesture. When operating in a calibration mode, femtoprojectors of a contact lens system can project images, such as matching pixels, symbols, or calibration patterns, and a user can provide an input to the alignment circuit in order to adjust the alignment of the femtoprojectors. For instance, a user can provide an input to adjust a location from which a first femtoprojector projects a test image until the test image is aligned with a test image projected by a second femtoprojector. The alignment circuit can, based on the user's input, determine alignment information for the femtoprojectors so that future images projected by the femtoprojectors are projected based on the alignment information.

Electronic Contact Lens Architecture

In more detail, FIG. 1A shows a user wearing an eye-mounted display 100 in communication with a necklace 160 (the user may be referred to as a wearer). FIG. 1B shows a magnified view of the user's eye and eye-mounted display 100 . The eye-mounted display 100 includes a contact lens 110 that is worn on the surface of the eye. The following examples use a scleral contact lens but the contact lens does not have to be scleral. The contact lens 110 contains multiple femtoprojectors 120 , some of which are central (projecting images on the wearer's fovea) and some of which are peripheral (projecting images adjacent to the image projected by the central projector). In FIG. 1B , the large center circle represents a central femtoprojector, and the six smaller surrounding circles each represent a peripheral femtoprojector. The femtoprojectors 120 project images onto the user's retina.

The contact lens 110 moves with the user's eye 100 as the user's eye rotates in its socket. Because the femtoprojectors 120 are mounted in the contact lens 110 , they also move with the user's eye. The central femtoprojector projects images to the fovea, and the peripheral femtoprojectors project images to more peripheral regions which have lower resolutions.

In this example, the contact lens 110 also contains electronics 140 and a coil 145 . In some embodiments, the coil 145 is a power coil that receives power wirelessly, for example via magnetic induction. In other embodiments, the contact lens 110 includes a battery that supplies power to the femtoprojectors 120 . The electronics 140 may be used to control the femtoprojectors, receive or process data from the femtoprojectors, provide power to the femtoprojectors, transmit data to the femtoprojectors, and/or align femtoprojectors. The contact lens 110 may also include other components.

FIG. 1A shows an implementation where, in addition to the eye-mounted display 100 , the user is also wearing a necklace 160 that contains components of the eye-mounted display system. In this example, the necklace 160 includes a wireless transmitter 170 that transmits image data and/or power to the eye-mounted display 100 . Image transmission to an eye-mounted display is subject to data rate constraints due to size and power consumption limitations of electronics in a contact lens.

FIG. 2A shows a cross sectional view of an eye-mounted display containing a set of femtoprojectors 210 - 220 in a scleral contact lens 110 . In this example, there are three types of femtoprojectors: a central femtoprojector 210 and two types of peripheral femtoprojectors 220 A and 220 B. The contact lens 110 is separated from the cornea 274 of the user's eye 270 by a tear layer 272 . Over the cornea 274 , the tear layer 272 may be as thick as a hundred microns or more while it may be only a few microns thick over the sclera 282 . The aqueous of the eyeball is located between the cornea and the crystalline lens 276 of the eye. The vitreous fills most of the eyeball including the volume between the crystalline lens 276 and the retina 278 . The iris 284 limits the aperture of the eye.

The contact lens 110 preferably has a thickness that is less than two mm, and the femtoprojectors 210 - 220 each preferably fits in a 2 mm by 2 mm by 2 mm volume, or less. The contact lens 110 is comfortable to wear and maintains eye health by permitting oxygen to reach the cornea 274 .

In addition, the contact lens includes an alignment circuit 230 configured to receive an input from a user and to align the femtoprojectors 210 - 220 based on the received input. The functionality of the alignment circuit 230 is described below in greater detail.

FIG. 2B shows a top view of the eye-mounted display with multiple femtoprojectors 210 - 220 in a contact lens 110 . The ratio of the contact lens diameter to femtoprojector lateral size may be 15:1 for the largest femtoprojector. This ratio may be between about 15:1 and 30:1, but may be as small as 5:1 or smaller or as large as 50:1 or larger. FIG. 2B shows three types of femtoprojectors; one central femtoprojector 210 , six “zone 1” peripheral femtoprojectors 220 A, and eight “zone 2” peripheral femtoprojectors 220 B. Eye-mounted displays with 49 or more femtoprojectors arranged in more than two zones in a contact lens are proposed. The femtoprojectors 210 - 220 in FIG. 2B are also shown as having different sizes.

In FIG. 2B , the contact lens 110 is roughly divided by the dashed circle into an optical zone 252 and a non-optical zone 254 . Components in the optical zone 252 may be in the optical path of the eye, depending on how far open the iris is. Components in the non-optical zone 254 fall outside the aperture of the eye. In addition to the femtoprojectors 210 - 220 , the contact lens may also contain other components for data transmission, power, and/or positioning. Data transmission components may include antennae or optical/infrared photodetectors, data storage and buffering, controls, and possibly also on-lens processing. Power components may include coils for power transmission and batteries for power storage. Positioning components may include accelerometers, magnetometers, gyroscopes, and fiducial or other structures used for eye tracking and head tracking.

FIG. 2C shows the images projected by the femtoprojectors onto the retina. Image 212 is the image projected by central femtoprojector 210 . Images 222 A are projected by the zone 1 peripheral femtoprojectors 220 A. Images 222 B are projected by the zone 2 peripheral femtoprojectors 220 B. Each group of peripheral femtoprojectors (the zone 1 femtoprojectors and the zone 2 femtoprojectors) project to an approximately annular region shown by the dashed circles.

In addition to the eye-mounted display, the overall system may also include a head tracker, eye tracker and other components. The system receives input images (including possibly video), which are to be displayed to the human user via the eye-mounted display. The femtoprojectors 210 - 220 project the images on the user's retina, thus creating an image of virtual objects in the user's field of view. Electronics and/or software receive the input images and produce the appropriate data and commands to drive the femtoprojectors. The head tracker and eye tracker provide information about head movement/position and eye movement/position, so that the information provided to the femtoprojectors can be compensated for these factors.

There are many ways in which this functionality can be configured with an eye-mounted display(s) to create embodiments of eye-mounted display systems. Portions of these subsystems may be external to the user, while other portions may be worn by the user in the form of a headpiece or glasses. Components may also be worn on a belt, armband, wrist piece, necklace, or other types of packs. FIG. 1A shows an example where some functionality is implemented in a necklace 160 worn by the user.

Projector Alignment

An alignment circuit 230 can align images projected by the femtoprojectors of the contact lens. The alignment circuit 230 may be a standalone circuit, or may be integrated into other contact lens components, such as the femtoprojectors 120 . The alignment circuit 230 can align images by laterally or rotationally moving an image relative to one or more other images. In some embodiments, the alignment circuit 230 aligns images such that the images overlap with each other. For instance, overlapping portions of two images projected by two femtoprojectors can be aligned such that the two images appear as one larger image. In other embodiments, the alignment circuit 230 can align images to be adjacent to each other, or to be spaced relative to each other. For instance, multiple femtoprojectors can each project a different and non-overlapping portion of an interface displayed to a user, and the femtoprojectors can be aligned to set a pre-determined spacing between the interface portions.

The alignment circuit 230 can adjust the location on the retina onto which an image is projected by adjusting one or both of the position and orientation of a femtoprojector relative to the contact lens. For example, the alignment circuit 230 can include an electroactive polymer or mechanical arm that, when activated, moves a location of the femtoprojector relative to the contact lens or moves an orientation or angle of the femtoprojector relative to the contact lens. The alignment circuit 230 can also adjust a location within a femtoprojector array from which an image is projected. The femtoprojector array is an area within a femtoprojector comprising multiple locations from which the femtoprojector can project images, such as a pixel grid. For instance, the alignment circuit 230 can provide a reference position within a femtoprojector array, and a femtoprojector can project an image from a position relative to the reference position (for instance, from a portion of the femtoprojector array centered on the reference position).

The alignment circuit 230 can align images based on an input from a wearer. For example, the wearer can provide an input using a smart device user interface, via a physical input device (e.g., a controller or other device), via a voice command, via a hand gesture or eye gesture, or via any other suitable input mechanism. In some embodiments, a user can provide an input moving a first image projected by a first femtoprojector until it is aligned with or adjacent to a second image projected by a second femtoprojector.

The alignment circuit 230 may align images when the contact lens is configured to operate in a calibration mode. The calibration mode may be initiated upon startup of the contact lens, in response to a request from the wearer, after the passage of a threshold period of time since a previous calibration, or in response to any other suitable criteria. When the contact lens is configured to operate in the calibration mode, the femtoprojectors may each project one or more test images to be aligned by the wearer. For example, a test image can include one or more alignment pixels, alignment symbols, or calibration patterns that the user can align via user input. For instance, each femtoprojector can project the same image, and the user can align the images until they appear as a single image. In another example, test images comprise different portions of a larger image that the user can align to form one larger image.

After the wearer aligns the test images, alignment information representative of the alignment can be stored by the contact lens. For instance, the alignment information can describe, for each femtoprojector, the location and/or orientation of the femtoprojector relative to the contact lens, the location within the femtoprojector array from which the aligned test image was projected, and the like. After the femtoprojectors are aligned, the contact lens can be configured to operate in a normal operating mode (for instance, in response to an input from the wearer indicating that the alignment or calibration is complete). During operation in the normal operating mode, the contact lens can configure the femtoprojectors to project images based on the alignment information. For instance, images can be projected from femtoprojectors at locations and orientations relative to the contact lens described by the alignment information.

FIGS. 3A and 3B show test images projected onto the retina by a set of femtoprojectors in a contact lens. The test images include image 305 and image 310 . Each test image includes a first alignment symbol 315 and a second alignment symbol 320 viewable by the wearer. In the example of FIG. 3A , the corresponding alignment symbols 315 are misaligned relative to each other by distance D 1 , and the corresponding alignment symbols 320 are misaligned relative to each other by distance D 2 .

The distances D 1 between the alignment symbols 315 and D 2 between the alignment symbols 320 can be reduced or eliminated by changing the locations on the retina onto which the images 305 and 310 are projected, and by rotating a first image relative to a second image. For instance, a user can provide an input requesting the image 310 be moved closer to the image 305 and rotated counterclockwise relative to the image 305 . In the example of FIG. 3B , the images 305 and 310 have been calibrated based on a user's input such that the alignment symbols 315 and the alignment symbols 320 each appear as a single alignment symbol.

FIGS. 4A-4C show a set of test images projected by a set of femtoprojectors within a contact lens. The images 405 a - e are each hexagonal portions of a larger image, and are each projected by a different femtoprojector. If the femtoprojectors are properly aligned, the images 405 appear as the single larger image 410 (as opposed to hexagonal portions of the image). Although the images 405 are hexagonal, it should be appreciated that in practice, the test images may be any shape or size.

In the example of FIG. 4A , the test images 405 are laterally misaligned relative to each other, and additionally, the test image 405 e is rotationally misaligned relative to the other test images. A wearer can provide an input requesting that the test image 405 e be rotated clockwise relative to the other test images. In response, the alignment circuit 230 can rotate the femtoprojector projecting the test image 405 e until it is no longer rotationally misaligned with the other test images as illustrated in the example of FIG. 4B .

The wearer can then provide inputs requesting that each of the test images 405 a - d be moved closer to test image 405 e , and the alignment circuit 230 can adjust a position within each femtoprojector array from which each of the test images 405 a - d are projected until they are adjacent to the test image 405 e . The example of FIG. 4C shows the test images 405 aligned to form a single larger image 410 . Although the test images 405 of FIGS. 4A-4C are intended to be displayed as a single larger image, in practice, some gaps between the test images may be preferred or intentional (for instance, as separate elements within an AR interface displayed by the femtoprojectors). In such instances, a wearer can align the test images 405 by positioning the test images relative to each other at desired spacing.

Although the detailed description contains many specifics, these should not be construed as limiting the scope of the invention but merely as illustrating different examples. It should be appreciated that the scope of the disclosure includes other embodiments not discussed in detail above. For example, the core may be other shapes, such as cylindrical or using polygon bases. Various other modifications, changes, and variations which will be apparent to those skilled in the art may be made in the arrangement, operation, and details of the method and apparatus disclosed herein without departing from the spirit and scope as defined in the appended claims. Therefore, the scope of the invention should be determined by the appended claims and their legal equivalents.