One-dimensional lens shading correction

Embodiments of the invention relate to a method and system to correct an image for artifacts due to lens shading.

In image capturing systems involving the lenses such as the one shown in FIG. 1 of the drawings, the light distribution by the lens system is non-uniform, causing a captured image to have some light intensity fall-off toward the edges of the image. This artifact is called the lens shading artifact or vignette. FIG. 2 of the drawings illustrates the effect of the lens shading artifact on a captured image. Referring to FIG. 2 , it will be seen that a captured image 10 includes a central circular region 12 . Light intensity is brightest at the center of the circular region 12 and falls off radially away from the center. Edges 14 at the corners at the image 10 show the greatest fall off in light intensity.

The lens shading artifact may be removed by multiplying the light intensity I(x,y) at the image location (x,y) by a lens shading correction amount L(x,y)

I corrected ( x,y )= I ( x,y )× L ( x,y )  (1)

In reality, for image capturing systems that include an image sensor, the sensor might contribute offset terms that would make the Equation (1) more complex. The offset terms are not considered so as to not to obscure the invention. However, one skilled in the art will know how to modify Equation (1) with the offset terms.

In one technique to correct for the lens shading artifact known to the inventor, the lens shading artifact is modeled so that the function L(x,y) may be inferred from the model. In another technique to correct for the lens shading artifact, the function L(x,y) is directly measured and calculated. With both these techniques, the function L(x,y) is a two dimensional (2D) function that depends on both x and y coordinates.

In one embodiment, a method for correcting image data for a captured image for lens shading artifacts is provided. The method comprises determining a plurality of one dimensional correction curves that can be applied to the image data to correct for lens shading artifacts therein; and correcting the image data for lens shading artifacts based on the one dimensional curves.

Other aspects of the invention will be apparent from the detailed description below.

Embodiments of the invention will now be described, by way of example, with reference to the accompanying diagrammatic drawings, in which:

FIG. 1 shows an image capturing system;

FIG. 2 illustrates the lens shading artifact;

FIG. 3 shows a high-level block diagram of a camera system, in accordance with one embodiment of the invention;

FIG. 4 shows a high-level block diagram of an image processor, in accordance with one embodiment of the invention;

FIG. 5 shows a flowchart for a lens shading correction technique; in accordance with one embodiment of the invention;

FIGS. 6 to 14 illustrate how lens shading correction techniques, in accordance with embodiments of the invention may be used to correct an image for lens shading artifacts.

In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art, that the invention may be practiced without these specific details. In other instances, structures and devices are shown at block diagram form only in order to avoid obscuring the invention.

Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not other embodiments.

Embodiments of the present invention disclose a method to correct lens shading artifacts by using two one dimensional (1D) functions L h (x)and L v (y) instead of a single 2D function L(x,y). An advantage of this method is that it is much easier and cheaper to compute two 1D functions than one 2D function. Moreover, if the functions are implemented as tables, then advantageously the present invention requires much less storage. For example, if the image size is 1000×1000, then two full 1D tables take 2×1000 or 2000 entries. On the other hand, the 2D table would take 1000×1000 or 1000000 entries.

Embodiments of the present invention also cover an image processor that corrects an image for lens shading artifacts contained therein, in accordance with the techniques of the present invention. Some embodiments also cover a camera system that includes such an image processor.

Turning now to FIG. 3 of the drawings, there is shown a camera system 20 in accordance with one embodiment of the invention. The camera system 20 includes camera optics 22 coupled to an image sensor 24 . The image sensor 24 is in turn coupled to an image processor 26 . As will be seen from FIG. 3 of the drawings, the image processor 26 includes lens shading correction logic 28 , and an image buffer 30 , among other components that have not been shown so as not to obscure the invention, but which components one of ordinary skill in the art would understand to form part of the image processor 26 . The lens shading correction logic 28 may be implemented in software, hardware, or a combination of both software and hardware.

In one embodiment, the lens shading correction logic 28 performs a method for correcting image data for a captured image for lens shading artifacts. This method is illustrated in the flowchart of FIG. 5 of the drawings. Referring to FIG. 5 , it will be seen that at block 32 , the lens shading correction logic 28 determines a plurality of one dimensional correction curves that can be applied to the image data to correct for lens shading artifacts therein. Thereafter, at block 34 the lens shading correction logic 28 corrects the image data for lens shading artifacts based on the one dimensional curves.

In accordance with one embodiment of the invention, operation of the lens shading correction logic 28 is based on the following equation that is modeled on Equation (1):

I corrected ( x,y )= I ( x,y )× C h ( x,y )× L h ( x )× C v ( x,y )× L v ( y )  (2)

In one embodiment both functions C h (x,y) and C v (x,y) are carefully selected so that they require very simple logic and very minimal storage to compute. Advantageously the computation of the Equation (2) is much simpler and cheaper than the Equation (1).

In accordance with the techniques of the present invention, a 2D image is corrected in two steps: one step makes the correction in the horizontal direction with the terms C h (x,y)×L h (x), and the second step makes the correction in the vertical direction with the terms C v (x,y)×L v (y). The horizontal correction and the vertical direction are treated independently of each other. Moreover, since the horizontal and the vertical corrections are similar, the logic for one correction may be reused for the other. In the description that follows, only the computation of C h (x,y) and L h (x) is described. The computation of C v (x,y) and L v (y) is not described as it is analogous to the computation of C h (x, y) and L h (x).

The Function L h (x)

For a uniform light source an image capturing system such as is shown in FIG. 1 of the drawings, will produce an image similar to that of FIG. 2 . For this image, for a line across the image horizontally at y coordinate y=y 0 as in FIG. 6 , a plot of the intensities of the pixels along the line versus the x positions of the pixels would result in a curve similar to that shown in FIG. 7

To perform a lens shading correction on the 1-dimensional data in FIG. 7 , the inverse of the curve is applied to the curve itself, thereby to straighten the curve. Since the data are taken by actual measurements, the data are not smooth. In one embodiment, noise in the data is removed by fitting a smooth curve to the data.

If the smooth curve is S(x), and its maximum value is S max then the correction curve should be

L y 0 ⁡ ( x ) = S max S ⁢ ⁢ ( x )

so that multiplying the correction curve L y 0 (x)with S(x) would yield a constant S max regardless of the position x. FIG. 8 shows the horizontal correction curve L y 0 (x) for the horizontal line at the y coordinate y=y 0 .

FIG. 9 shows the horizontally corrected data for the horizontal line at the y coordinate y=y 0 .

If the process is repeated at a different y coordinate, say y=y i , then the function L y i (x) for the new horizontal line y=y i should be different than that at the line y=y 0 as shown in FIG. 10 . However, the two functions are very similar, and in one embodiment this similarity is the basis for the simplification of the function C h (x,y). In one embodiment, C h (x,y)is selected so that it “morphs” the curve L y 0 (x) into L y i (x) as follows:

C h ( x,y i )× L y 0 ( x )= L y i ( x )

At the center of the lens, the light intensity is brightest and there is no need to apply the lens shading correction at the center. Thus, at the center of the lens L h (x)=1. However, in real applications, the center of the lens is not always at the center of the image. Thus, in one embodiment the center of the lens is determined by looking for the brightest spot in the captured image of a uniform light. If that position is (x 0 , y 0 ), then

L h ( x )= L y 0 ( x ) and L v ( y )= L x 0 ( y )

Thus, L h (x) and L v (y) are lens shading correction curves running through the brightest point of the image.

The Function C h (x,y)

Theoretically, the function C h (x,y) may be computed so that C h (x,y i )×L y 0 (x)=L y i (x), but in reality, computing C h (x,y) in this manner is computationally more expensive than the original 2D correction function L(x, y). Thus, in one embodiment of the invention C h (x,y) is not calculated exactly, but instead C h (x,y) is approximated.

In one embodiment, the function L y i (x) at the y coordinate y=y i from the captured image data is approximated from the function L y 0 (x) as follows:

1. First, the source correction function L y 0 (x) is divided into two segments at the lowest point. 2. The target correction function L y i (x) is also cut into two segments at the lowest point. 3. The first segment is morphed or transformed from the source correction function into the target correction function by scaling the first end point in the vertical direction, and the second end point in the horizontal direction. 4. Similarly, the second segment is morphed from the source correction function into the target correction function by scaling the first end point in the horizontal direction, and the second end point in the vertical direction.

FIG. 11 illustrates the transformation or morphing of the two functions. Using this morphing technique, it is not necessary to store or model the target correction function. Only the scaling parameters are stored. Thus, storage required to implement the techniques of the present invention is minimal. The scaling computation is not complex as it requires only additions and multiplications.

For a perfectly aligned lens, the lowest points of the source and target functions are the same, so the horizontal scaling is not necessary. Furthermore, the left and right end points of the correction curve usually need to be bent uniformly. For such a lens, duplicated or unused scaling parameters may be eliminated, in one embodiment.

For a tilted lens, the lens shading artifact has an elliptical shape. In this case, the tilted angle normally does not align with the edges of the image sensor, so the lowest points of the source and target functions are different as shown by the two X's in FIG. 12 . This problem may be overcome by cutting the correction curves at the lowest points, and treating them as two different segments. For misaligned and tilted lens, the two end points also bend differently, but again, segmenting the curve and bending the segments differently solves the problem.

For some lens systems with multiple lenses, the source and target correction functions might differ a lot at the ends as can be seen from FIG. 13 of the drawings. In FIG. 13 , it is to be observed that one of the curve bends down toward the ends, while the other one does not. To fix this problem, in one embodiment, the correction curves are cut into more segments, each of which is morphed independently as shown in FIG. 14 .

As discussed earlier, the morphing reduces the complexity of the function L y i (x) into a few scaling parameters. This reduces the storage and computation for the line y=y i . In one embodiment, this process may be repeated for every line in the image. In another embodiment, the scaling parameters between the lines are morphed, for example by linear interpolation. Thus, in one embodiment only the scaling parameters for the top and bottom rows are stored. For each row between the top row and the row y=y 0 , the scaling parameter is interpolated between the value for the top row and a value of 1 (the scaling parameters for the row y=y 0 ). Similarly, for each row between y=y 0 and the bottom row, the scaling parameter is interpolated between a value of 1 and the value for the bottom row

Even though the horizontal and vertical lens shading corrections are two independent steps, care must be taken to avoid one from overcorrecting the image after the other. Thus, for example, in one embodiment if C h (x,y) and L h (x) are computed first, then C h (x,y) and L h (x) are applied first to the image, then C v (x,y) and L v (y) are applied to the corrected image, instead of the original one.

Digital cameras typically capture only one color per pixel. To be able to reconstruct the captured data into color images, the sensor pixels are divided into 3 or 4 different colors. Because the lens shading corrections are usually different for different captured colors, embodiments of this invention may be applied to color images by using 3 or 4 independent sets of lens shading correction functions.