Methods and apparatus for a multiple storage pixel imaging system

Multi-storage node pixels may be beneficial in high dynamic range (HDR) imaging, which may include overlapped-exposure imaging techniques. In overlapped-exposure HDR imaging, multiple images are captured with an image sensor over the same time period but with different exposure periods, and the images are later combined into a high dynamic range image.

Images captured at different times for the purpose of HDR imaging may result in distorted images for moving objects. Capturing images in an overlapped manner reduces the distortion as multiple image captures receive the same motion at the same time.

Rolling shutter architectures also result in image distortion for moving objects since images are captured in a sequential row-by-row fashion. As such, a global shutter design may be preferred to minimize the motion distortion associated with rolling shutter circuits, since all pixels in a global shutter imager integrate light simultaneously.

Conventional global shutter pixels capable of correlated doubled sampling have at least one photo sensitive element, a dedicated in-pixel storage region, and a floating diffusion region. The pixels operate by transferring charge to the storage region after integration, resetting the floating diffusion region prior to readout, and then reading out the charge from the storage region to the floating diffusion region for sampling. As such, the floating diffusion region is utilized only during the readout period.

Creating a HDR image utilizing a global shutter design with pixels formed with multiple storage regions results in larger pixels, or for a given pixel size, reduces the area for the photodiode, thus decreasing sensitivity.

A more complete understanding of the present technology may be derived by referring to the detailed description when considered in connection with the following illustrative figures. In the following figures, like reference numbers refer to similar elements and steps throughout the figures.

FIG. 1 representatively illustrates a multi-storage pixel in accordance with an exemplary embodiment of the present technology;

FIG. 2 representatively illustrates a timing diagram in accordance with an exemplary embodiment of the present technology;

FIG. 3 representatively illustrates an image sensor in accordance with an exemplary embodiment of the present technology; and

FIG. 4 representatively illustrates an imaging system in accordance with an exemplary embodiment of the present technology.

The present technology may be described in terms of functional block components and various processing steps. Such functional blocks may be realized by any number of components configured to perform the specified functions and achieve the various results. For example, the present technology may employ various semiconductor devices, such as switches, capacitors, photosensors, and the like, which may carry out a variety of functions. In addition, the present technology may be practiced in conjunction with any number of imaging systems and the apparatus and method embodiments described are merely exemplary applications for the technology. Further, the present technology may employ any number of conventional techniques for capturing image data, sampling image data, readout of image data, and the like.

Methods and apparatus for a multi-storage node pixel 100 ( FIG. 1 ) imaging system according to various aspects of the present technology may operate in conjunction with any suitable imaging system, such as a camera system, video system, machine vision, vehicle navigation, surveillance system, motion detection system, image stabilization system, and the like. For example, referring to FIG. 4 , the imaging system may comprise an electronic device, such as a digital camera 400 . The system may comprise a central processing unit (CPU) 405 that communicates with various devices over a bus 410 . Some of the devices connected to the bus 410 may provide communication into and out of the system, for example an input/output (I/O) device 415 . Other devices connected to the bus 410 provide memory, for example, a random access memory (RAM) 420 , hard drive, and one or more peripheral memory devices 425 , such as a floppy disk drive, compact disk (CD) drive, USB drives, memory cards, and SD cards. While the bus 410 is illustrated as a single bus, any number of busses may be used to provide communication paths to interconnect the devices.

The imaging system may further comprise an imaging sensor for capturing image data. For example, light may enter the camera through a lens 430 and strike the image sensor. The image sensor may detect and convey the information that constitutes an image, for example by converting the variable attenuation of waves (as they pass through or reflect off objects) into electronic signals. The image sensor may be implemented in conjunction with any appropriate technology, such as using semiconductor charge-coupled devices (CCD), active pixel sensors in complementary metal-oxide-semiconductors (CMOS) or N-type metal-oxide-semiconductors (NMOS), analog sensors, and/or flat panel detectors.

Referring to FIG. 3 , an exemplary image sensor 300 may comprise a pixel array 305 comprising a plurality of multi-storage pixels 100 ( FIG. 1 ) arranged in rows and columns. The pixels 100 may be electrically connected via metal wirings or other suitable connections. The image sensor 300 may be formed in silicon using any suitable complementary metal-oxide semiconductor (CMOS) technology and/or fabrication process.

In various embodiments, the image sensor 300 may further comprise a processing circuit 310 . The processing circuit 310 may be coupled to the pixel array 305 via a communication bus 315 , wherein the communication bus 315 may transmit and receive signals, such as control and data signals, between the pixel array 305 and the processing circuit 310 . The processing circuit 310 may comprise a control unit (not shown) for transmitting control signals. The processing circuit 310 may also comprise a clocking mechanism to synchronize transmission of control signals. The processing circuit 310 may read the pixel signals through control lines accessed in a per-row manner. In various embodiments, the processing circuit 310 may be formed on the same chip as the pixel array 305 .

Each multi-storage pixel 100 of the image sensor 300 samples a portion of the image and generates a corresponding signal. Referring to FIG. 1 , in an exemplary embodiment of the present technology, the multi-storage pixel 100 may comprise a photosensitive element 105 , a storage node 110 , and a floating diffusion node 115 . In the present embodiment, the photosensitive element 105 operates by converting light into an electric charge and may comprise a photodiode, a photogate, or other semiconductor device responsive to light. In various embodiments, the photosensitive element 105 may comprise a pinned photodiode capable of being fully depleted at a depletion voltage.

The floating diffusion node 115 (also referred to as a “floating diffusion region” or “FD node”) selectively stores charge. In the present embodiment, the floating diffusion node 115 may act as a sensing node and may be formed with any device or structure suitable for storing electric charge, such as a diode or capacitor. In various embodiments, the floating diffusion region 115 may be shielded from incoming light.

The storage node 110 (also referred to as a “storage region”) selectively stores charge. In the present embodiment, the storage node 110 may comprise a doped region into which electric charge is transferred and stored. The storage node 110 may be formed with any device or structure suitable for storing electric charge, for example, a pinned diode or a storage gate. In various embodiments, the storage region 110 may be shielded from incoming light.

The photosensitive element 105 may be selectively coupled to the floating diffusion node 115 via a first transfer gate 120 . In an exemplary embodiment, the first transfer gate 120 may comprise a switch, such as a transistor having a gate terminal, a drain terminal, and a source terminal, wherein the gate terminal may operate as a control terminal, and the source and drain terminals are used to carry current or transfer charge. For example, the gate terminal of the first transfer gate 120 may receive a first control signal TX 1 from the processing circuit 310 ( FIG. 3 ). In other embodiments, the first transfer gate 120 may comprise any other suitable device for providing current flow or charge transfer.

The floating diffusion node 115 may also be selectively coupled to a voltage source V AA , for example via a switch, such as a reset gate 135 . In the present embodiment, the reset gate 135 may comprise a transistor having a gate terminal, a drain terminal, and a source terminal, wherein the gate terminal may operate as a control terminal, and the source and drain terminals carry current when the reset gate 135 is activated. For example, the gate terminal of the reset gate 135 may receive a control signal RST from the processing circuit 310 ( FIG. 3 ).

The photosensitive element 105 may be selectively coupled to the storage node 110 via another switch, such as a second transfer gate 125 . In the present embodiment, the second transfer gate 125 may comprise a transistor having a gate terminal, a drain terminal, and a source terminal, wherein the gate terminal may operate as a control terminal and the source and drain terminals carry current or transfer charge. For example, the gate terminal of the second transfer gate 125 may receive a second control signal TX 2 from the processing circuit 310 ( FIG. 3 ). In other embodiments, the second transfer gate 125 may comprise any other suitable device for controlling current flow or charge transfer.

The storage node 110 may be selectively coupled to the floating diffusion node 115 via a switch such as third transfer gate 130 . In the present embodiment, the third transfer gate 130 may comprise a transistor having a gate terminal, a drain terminal, and a source terminal, wherein the gate terminal may operate as a control terminal and the source and drain terminals carry current or transfer charge. For example, the gate terminal of the third transfer gate 130 may receive a third control signal TX 3 from the processing circuit 310 ( FIG. 3 ). In other embodiments, the third transfer gate 130 may comprise any other suitable device for providing current flow or charge transfer.

The pixel 100 may further comprise an amplifier 140 and a row select gate 150 . The amplifier 140 amplifies the signal generated by the photosensitive element 105 . For example, in the present embodiment, the amplifier 140 may comprise a source follower circuit comprising a transistor having a gate terminal, a drain terminal, and a source terminal. The gate terminal may receive signals corresponding to the charge generated by the photosensitive element and provide a corresponding amplified signal.

The row select gate 150 selectively connects the pixel 100 to the output. Each line in the imager sensor 300 may be selected and then read out, for example using a column select signal. The row select gate 150 for each pixel in a row may be activated to make that row active. In the present embodiment, the row select gate may comprise a transistor having a gate terminal, a drain terminal, and a source terminal. The gate terminal may be used as a control terminal and the source and drain terminals carry current. The gate terminal of the row select gate 150 may receive a control signal RS from the processing circuit 310 ( FIG. 3 ). In an exemplary embodiment, the source terminal of the amplifier 140 may couple to the drain terminal of the row select gate 150 .

Referring now to FIGS. 1 and 2 , in operation 200 , the image sensor 300 ( FIG. 3 ) may operate in conjunction with a global reset period 205 , an integration period 210 , and a readout period 215 . In an exemplary embodiment, the image sensor 300 may capture multiple signals with different exposure periods.

In an exemplary embodiment, during the global reset period 205 , the first, second, and third transfer gates 120 , 125 , 130 , as well as the reset gate 135 , are simultaneously active. For example, control signals TX 1 , TX 2 , TX 3 , and RST are switched to a high voltage value simultaneously. Transmitting a control signal with a high voltage value to the first, second, and third transfer gates 120 , 125 , 130 , and reset gate 135 resets the photosensitive element 105 . After a predetermined period of time, the first, second, and third transfer gates 120 , 125 , 130 , and reset gate 135 may be deactivated, for example via low voltage control signals. In an exemplary embodiment, all pixels 100 in the pixel array 305 ( FIG. 3 ) receive the same control signals and are reset simultaneously.

After the reset period 205 has been completed, the global integration period 210 begins. During integration, the photosensitive element 105 absorbs light and converts the light into an electric charge. In an exemplary embodiment, the electric charge produced in the photosensitive element 105 is transferred to both the storage node 110 and the floating diffusion node 115 . Charge transfer from the photosensitive element 105 to the storage node 110 and the floating diffusion node 115 may be controlled by selectively activating control signals TX 1 and TX 2 by pulsing the signals high at different times, wherein the pulses for control signals TX 1 and TX 2 do not overlap. For example, control signals TX 1 and TX 2 do not activate the first and second transfer gates 120 , 125 at the same time. Instead, only one of the first and second transfer gates 120 , 125 is activated at any given time. In an exemplary embodiment, the integration period may comprise one or more exposures.

After a first exposure period T 1 , typically measured in micro seconds (μs), the first transfer gate 120 may be activated, for example via a high voltage value control signal TX 1 , allowing the charge accumulated in the photosensitive element 105 to transfer to the floating diffusion node 115 . The first transfer gate 120 may be deactivated, such as via a low voltage value control signal TX 1 , to stop charge transfer to the floating diffusion node 115 . After a second exposure period T 2 , measured in microseconds (μs), a high voltage value control signal TX 2 activates the second transfer gate 125 , allowing the charge accumulated in the photosensitive region 105 to transfer to the storage node 110 . A low voltage value control signal TX 2 stops charge transfer to the storage node 110 . Charge transfer to the floating diffusion node 115 and the storage node 110 may begin in any order. For example, at the start of the integration period 210 , charge may be transferred and stored in either one of the floating diffusion node 115 or the storage node 110 . As such, the second exposure period T 2 may occur earlier in time than the first exposure period T 1 .

In an exemplary embodiment, during the integration period 210 , control signals TX 1 and TX 2 may be asserted multiple times such that the charges transferred the floating diffusion node 115 are summed. Likewise, the charges transferred to the storage region 110 are summed (binned). Operation of the control signals TX 1 and TX 2 may be controlled by the processing circuit 310 ( FIG. 3 ) and may operate cyclically, such that charge is transferred from the photosensitive element 105 to the floating diffusion node 115 and then, immediately after another exposure period, charge is transferred from the photosensitive element 105 to the storage region 110 . In other embodiments, charge may not be transferred in a cyclical manner, but may be transferred in any suitable pattern. For example, charge may be transferred consecutively to the storage region 110 for any number suitable of times; likewise, charge may be transferred consecutively to the floating diffusion node 115 for any suitable number of times.

In an exemplary embodiment, where the integration period comprises multiple exposure periods T 1 i , T 2 i , the charge portions transferred to the storage region 110 may accumulate and are summed. Likewise, the charge portions transferred to the floating diffusion node 115 may accumulate and are summed. The transfer of multiple charge portions with corresponding exposure periods T 1 i , T 2 i to each of the storage region 110 and the floating diffusion node 115 may result in the accumulated charges having different total exposure times T 1 total , T 2 total .

In an exemplary embodiment, the first exposure period T 1 may be shorter than the second exposure period T 2 . A long exposure period may be used to capture low-light portion of a scene, while a short exposure period may be used to capture bright portions of the scene. For example, the charge transferred to the floating diffusion node 115 after the first exposure period T 1 captures bright scenes (i.e. high light signals), and the charge transferred to the storage node 110 after the second exposure period T 2 , where T 2 is greater than T 1 , captures low-light scenes (i.e. low light signals). As such, the total exposure time for charge accumulated in the storage node 110 may be longer than that of the total exposure time for charge accumulated in the floating diffusion node 115 . The total exposure times may be described by the following equations, where n and m are the upper limits of the number of exposures and i is the index number for each exposure.

T 1 total =Σ i=1 n T 1 i

T 2 total =Σ i=1 m T 2 i

In various embodiments, n may be equal to m, n may be greater than m, or n may be less than m. The upper limits of the number of exposures n, m may be selected to suit a particular application. Additionally, the length of the exposure periods T 1 i , T 2 i may vary during any one integration period 210 , for example, T 1 1 may be 50 μs (microseconds), while T 1 10 may be 400 μs.

Dynamic range of the image sensor may be defined as the ratio of the sum of second exposure periods T 2 to the sum of the first exposure periods T 1 (i.e. T 2 total /T 1 total ). The dynamic range may be adjusted during operation by increasing and/or decreasing the length of any of the exposures times T 1 i , T 2 i . The total exposure times T 1 total , T 2 total may be tracked in real time such that a running total may be computed during the integration period 210 and transmitted to the control unit, where the control unit may adjust the duration of any subsequent exposure periods T 1 i , T 2 i .

Pulsing of the control signals TX 1 , TX 2 may be represented as a duty cycle, where the duty cycle is defined as a length of time where the signal is pulsed high T h , to activate the transfer gates 120 , 125 divided by the total amount of time in a cycle T c . In various embodiments, exposure periods T 1 i , T 2 i may be predetermined, while in other embodiments, the exposure periods T 1 i , T 2 i may be varied, resulting in a varied duty cycle. For example, the duty cycle and frequency of a voltage value applied to the control signals TX 1 and TX 2 may be synchronized to a pulsing light source for time of flight applications. The duty cycle and frequency of a voltage value applied to the control signals TX 1 and TX 2 may also be used to mitigate amplitude modulated light sources, such as LEDs (light emitting diodes).

Once the integration period 210 ends, the readout period 215 begins. In an exemplary embodiment, the pixel signals are read out on a row-by-row basis. A high voltage value is applied to the row select control signal RS, and in an exemplary embodiment, the row select control signal RS maintains a high voltage value for the duration of the readout period 215 for each row. Applying a high voltage value to the row select control signal RS allows the charge that has accumulated in the floating diffusion node 115 to be read out of the pixel 100 via a communication line 155 as a voltage which is proportional to the amount of accumulated charge. The readout process produces a first pixel signal comprising a corresponding voltage. The first pixel signal may be unable to undergo correlated double sampling since the floating diffusion reset voltage cannot be measured prior to readout. However, the first pixel signal can be double sampled, once before the reset gate 135 is activated 220 and once using the voltage level post reset 225 .

While the row select control signal RS maintains a high voltage value, the reset control signal RST activates the reset gate 135 and resets the floating diffusion node 115 . At this time 225 , the floating diffusion reset level can be sampled and used for double sampling of the first pixel signal, and used for correlated double sampling of the second pixel signal.

Once the floating diffusion node 115 has been reset and the reset level sampled, the charge that has accumulated in the storage node 110 can be read out. For example, activating the third transfer gate 130 via control signal TX 3 allows the charge to transfer from the storage node 110 through the floating diffusion node 115 , and the resulting voltage of pixel 100 is accessible via the communication line 155 . The readout produces a second pixel signal comprising a corresponding voltage. In various embodiments, as described above, the second pixel signal read out from the storage node 110 may undergo correlated double sampling to reduce noise since the voltage on the floating diffusion node 115 may be measured twice, once in a reset condition 225 (prior to activating the third transfer gate 130 ), and once after the charge from the storage node 110 has been transferred to the floating diffusion node 115 , 230 .

After both pixel signals have been read out of the floating diffusion node 115 and the storage node 110 , the row select control signal applies a low voltage value to end the readout period 215 . A new global reset period 205 may begin again.

In an exemplary embodiment, after completion of the readout period 215 , the pixel signals (i.e. one capture using a long exposure and one capture using a short exposure) may be combined to produce a high dynamic range image utilizing an image signal processor (not shown). In various embodiments, the image signal processor may also perform color processing and tone mapping operations, such as demosaicing, white-balance, and color correction.

The embodiments of FIGS. 2 and 3 in which the pixel array 305 is described as operating in global shutter mode is merely illustrative. If desired, the pixel array 305 having the processing circuit 310 for operating in global shutter mode may be operated in rolling shutter mode. The particular mode of operation depends on the timing and operation of the control signals control signals TX 1 , TX 2 , TX 3 , RST, and RS. For example, in rolling shutter mode, the signals controlling the integration 210 readout 215 , and reset 205 of each pixel 100 would be operated sequentially in a row-by-row fashion.

Additionally, the terms “first” and “second” do not connote a particular order in time, but rather distinguish between distinct elements, events, conditions, and the like. For example, the second exposure period T 2 may occur earlier in time than the first exposure period T 1 .

In the foregoing description, the technology has been described with reference to specific exemplary embodiments. The particular implementations shown and described are illustrative of the technology and its best mode and are not intended to otherwise limit the scope of the present technology in any way. Indeed, for the sake of brevity, conventional manufacturing, connection, preparation, and other functional aspects of the method and system may not be described in detail. Furthermore, the connecting lines shown in the various figures are intended to represent exemplary functional relationships and/or steps between the various elements. Many alternative or additional functional relationships or physical connections may be present in a practical system.

While the technology has been described with reference to specific exemplary embodiments, various modifications and changes may be made without departing from the scope of the present technology. The description and figures are to be regarded in an illustrative manner, rather than a restrictive one and all such modifications are intended to be included within the scope of the present technology. Accordingly, the scope of the technology should be determined by the generic embodiments described and their legal equivalents rather than by merely the specific examples described above. For example, the steps recited in any method or process embodiment may be executed in any order, unless otherwise expressly specified, and are not limited to the explicit order presented in the specific examples. Additionally, the components and/or elements recited in any apparatus embodiment may be assembled or otherwise operationally configured in a variety of permutations to produce substantially the same result as the present technology and are accordingly not limited to the specific configuration recited in the specific examples.

Benefits, other advantages and solutions to problems have been described above with regard to particular embodiments. Any benefit, advantage, solution to problems or any element that may cause any particular benefit, advantage or solution to occur or to become more pronounced, however, is not to be construed as a critical, required or essential feature or component.

The terms “comprises”, “comprising”, or any variation thereof, are intended to reference a non-exclusive inclusion, such that a process, method, article, composition or apparatus that comprises a list of elements does not include only those elements recited, but may also include other elements not expressly listed or inherent to such process, method, article, composition or apparatus. Other combinations and/or modifications of the above-described structures, arrangements, applications, proportions, elements, materials or components used in the practice of the present technology, in addition to those not specifically recited, may be varied or otherwise particularly adapted to specific environments, manufacturing specifications, design parameters or other operating requirements without departing from the general principles of the same.

The present technology has been described above with reference to an exemplary embodiment. However, changes and modifications may be made to the exemplary embodiment without departing from the scope of the present technology. These and other changes or modifications are intended to be included within the scope of the present technology, as expressed in the following claims.