Method for enhancing digital images

The present invention relates to image processing in a computing system and more particularly, to a method for enhancing the quality of digital images.

Digital images are commonly used in today's computing world. However, the quality of these images vary widely. For example, some digital images are highly compressed JPEG files with artifacts, but others are of a very high quality. The inherent variation in the quality of digital images poses a challenge for image rendering devices, such as printers and computer monitors to accurately reproduce digital images.

Prior image processing techniques have been proposed for improving the quality of digital images. Conventional algorithms have been used for suppressing compression artifacts and also for restoring images that are distorted by various types of noise. However, applying an algorithm for suppressing compression artifacts to an image that does not have any artifacts will result in a degraded image. For example, if an algorithm for reducing JPEG artifacts is applied to a sharp image that does not have JPEG artifacts, it will result in a blurry image compared to the original image.

Other conventional image processing techniques have been proposed for enhancing images without excessive artifacts. Examples of such techniques are, unsharp masking algorithms for image sharpening and histogram transformations for enhancing image contrast. However, application of the foregoing processes to an image that does include artifacts will also result in image distortion.

Therefore, what is desired is a method for improving the quality of digital images with or without artifacts resulting in an enhanced image.

The present invention addresses the foregoing by providing a method for improving arbitrary digital images with an input pixel window having an input pixel, including process steps to calculate variation in luminance of the pixel window, compare the variation in the luminance of the pixel window to a predetermined first threshold value and process the input pixel based upon the comparison. Thereafter, output pixel with pixel value equal to the input pixel values, if the luminance is less than the first threshold value; apply a moderate smoothing filter if the variation in luminance is greater than the first threshold value and less than a second predetermined threshold value; and apply a non-linear, enhancement filter if the luminance variation is greater than the second threshold value.

By virtue of the foregoing aspects of the present invention, a filter is applied to a digital image based upon image content and that improves the quality of reproduced digital images.

This brief summary has been provided so that the nature of the invention may be understood quickly. A more complete understanding of the invention can be obtained by reference to the following detailed description of the preferred embodiments thereof in connection with the attached drawings.

FIG. 1 illustrates a computing system to carry out the inventive technique.

FIG. 2 is a block diagram of the architecture of the computing system of FIG. 1 .

FIG. 3 illustrate a raster image with an input pixel window.

FIG. 4 is a basic flow chart showing computer executable process steps for enhancing arbitrary digital images.

FIG. 5 is a basic flow chart showing computer executable process steps for applying a moderate smoothing filter to an input pixel window.

FIG. 6 is a flow diagram showing computer executable process steps for applying a non-linear enhancement filter to an input pixel window.

FIG. 7 is a flow diagram showing computer executable process steps for selecting filter coefficients used in the filter of FIG. 6 .

FIG. 1 is a block diagram of a typical printing system, which includes a host computer 10 , a monitor 11 , and a printer 12 . Monitor 11 may be a CRT type, a LCD type, or any other type of color or monochrome display. Printer 12 may be any type of printer such as an inkjet printer, laser printer, thermal printer, dot matrix, or the like. Also provided with computer 10 is a keyboard 13 for entering text data and user commands, and a pointing device 14 for processing objects displayed on monitor 11 .

Computer 10 includes a computer-readable memory medium such as a rotating disk 15 for storing readable data. Besides other programs, disk 15 stores computer executable code embodying the present invention, and application programs by which computer 10 generates, manipulates and stores files on disk 15 , displays data in those files on monitor 11 , and prints data in those files via printer 12 .

Device drivers are also stored on disk 15 . One of the stored device drivers is a printer driver that provides a software interface to firmware in printer 12 , thereby facilitating data transfer between computer 10 and printer 12 .

Computer 10 can also access a computer-readable floppy disk storing data files, application program files, and computer executable process steps embodying the present invention or the like via a floppy disk drive 16 . A CD-ROM interface (not shown) may also be provided with computer 10 to access application program files, device driver files and data files stored on a CD-ROM.

A modem, an integrated services digital network (ISDN) connection, or the like also provides computer 10 with an Internet connection 17 to the World Wide Web (WWW). The Internet connection 17 allows computer 10 to download device drivers, data files, image files, application program files, and computer-executable process steps embodying the present invention.

FIG. 2 is a block diagram showing the internal functional architecture of computer 10 . As shown in FIG. 2, computer 10 includes a CPU 201 for executing computer-executable process steps and interfaces with a computer bus 209 . Also shown in FIG. 2 are a printer interface 202 , a WWW interface 203 , a display device interface 204 , a keyboard interface 205 , a pointing device interface 206 and disk 15 .

As described above, disk 15 stores operating system program files, application program files, and device drivers such as a printer driver for printer 12 . Some of these files are stored on disk 15 using an installation program. For example, CPU 201 executes computer-executable process steps of an installation program so that CPU 201 can properly execute the application program.

A random access main memory (“RAM”) 207 also interfaces to computer bus 209 to provide CPU 201 with access to memory storage. When executing stored computer-executable process steps from disk 15 (or other storage media such as floppy disk 16 , or from files download via WWW connection 17 ), CPU 201 stores those process steps in RAM 207 and executes the stored process steps out of RAM 207 .

Read only memory (“ROM”) 208 is provided to store invariant instruction sequences such as start-up instruction sequences or basic Input/output operating system (BIOS) sequences for operation of keyboard 13 .

FIG. 3 is a representational view of a raster digital image 30 with input pixel window 31 . As shown in FIG. 3, pixel window 31 comprises of pixels arranged in a two dimensional array. Each pixel of the raster image 30 has a color value which is stored in memory such as on disk 15 or in RAM 207 . The number of bits for the color value is determined by the color space, for example, RGB, CMYK, LAB, and CIELAB etc. Frequently color is represented by 24 bits, that includes 8 bits for each of red, green and blue color. Furthermore, color can be represented in the CMYK-color space by 32 bits which includes 8 bits for each cyan, magenta, yellow and black color components. For illustration purposes, also shown in FIG. 3 is an input pixel 32 located in pixel window 31 whose output value (“output pixel 33 ”) is determined according to one aspect of the present invention.

FIG. 4 is a flow diagram showing process steps for enhancing arbitrary digital images according to one aspect of the present invention. The process steps of FIG. 4 are stored as computer code in RAM 207 and executed by CPU 201 or alternatively, stored on disk 15 and executed by CPU 201 . Generally speaking, the process depicted in FIG. 4 analyzes the variation in input pixel window 31 , applies a filter based upon the variation and outputs pixel values for an input pixel 32 .

In more detail, step S 401 obtains an input pixel window. For illustration purposes as shown in FIG. 3, input pixel window 31 is a 5×5 array comprising of 25 pixels with pixel P 13 , designated as the input pixel 32 .

In step S 402 , calculate luminance for pixel window 31 . The luminance (L) can be calculated by using the formula: L=0.299R+0.587G+0.114B where R, G and B are the red, blue and green components of the pixels. Luminance can also be calculated by other formulas based upon the color space.

Step S 403 , determines variation in input pixel window 31 . Variation in input pixel window 31 may be determined by the difference between the maximum and minimum value of luminance of pixel window 31 . Other factors, for example, hue and saturation can also be used to determine the variation in input pixel window 31 .

Step 404 , compares luminance variation of pixel window 31 to a predetermined threshold value (T 1 ). If luminance variation is less than T 1 , then the process flow moves to Step S 408 , where input pixel value is equal to output pixel value.

If luminance variation is greater than T 1 , then in step S 405 , luminance variation is compared to a second predetermined threshold (T 2 ). If luminance variation is less than T 2 , then in Step S 406 a moderate smoothing filter as described in FIG. 5 is applied to input pixel window 31 to generate output pixel 33 . If variation is greater than T 2 , then the process flow moves to step S 407 , where a non-linear enhancement filter, as described in FIG. 6 is applied to input pixel window 31 to generate output pixel 33 .

Applying a Moderate Smoothing Filter

In Step S 501 , input pixel window 31 is received from Step S 406 , as described in FIG. 4 .

In Step S 502 , apply a moderate smoothing filter to the red plane of pixel window 31 . One such moderate smoothing filter with the coefficients given below are applied to a 3×3 array, where input pixel 32 is the center of the 3×3 array, as shown in FIG. 3 .

Moderate Smoothing Filter Coefficients:

1/9, 1/9, 1/9

1/9, 1/9, 1/9

1/9, 1/9, 1/9

It is noteworthy, that the foregoing moderate smoothing filter coefficients are for illustration purposes only, another set of coefficients may be used to apply a moderate smoothing filter to input pixel window 31 . Furthermore, the invention is not limited to a 3×3 array, any array size may be used for applying the moderate smoothing filter.

In Step 503 , output red coordinate of output pixel 33 .

In Step S 504 , apply a moderate smoothing filter to the green plane of pixel window 31 similar to the red plane discussed above. In Step S 505 output green coordinate of output pixel 33 .

In Step S 506 , apply a moderate smoothing filter to the blue plane of pixel window 31 similar to the red plane discussed above.

In Step S 507 , output the blue coordinate of output pixel 33 .

Applying a Non-linear Enhancement Filter

FIG. 6 is a block diagram of computer executable process steps according to one aspect of the present invention for applying a non-linear enhancement filter when the variation in luminance of pixel window 31 is greater than T 2 , as shown in FIG. 4 .

In step S 601 , input pixel window 31 received from Step S 407 of FIG. 4 .

In step S 602 , select filter coefficients as described in detail in FIG. 7 .

In step S 603 , calculate variation in the red plane of pixel window 31 . The variation in the red plane can be calculated by the difference between the maximum and minimum value of the red plane.

In Step S 604 , compare the variation of the red plane to a predetermined threshold value for the red plane, designated as T r .

In Step S 605 , determine if variation in the red plane is greater than T r . If variation in red plane is less than T r then in step S 606 , the red coordinate of output pixel 33 is equal to the red coordinate of input pixel 32 . If variation in red plane is greater than T r , then the process moves to Step S 607 and filter coefficients obtained from FIG. 7 are applied to the red plane of the input pixel window 33 .

In Step S 608 , output red-coordinate value for output pixel 33 .

Steps similar to S 603 -S 608 are applied to the green and blue planes of pixel window 31 to generate green and blue coordinates for output pixel 33 based on input pixel 32 .

Determining Filter Coefficients

FIG. 7 is a block diagram of computer executable process steps according to one aspect of the present invention that describes filter selection process used for applying filter coefficients in FIG. 6 . Generally, FIG. 7 describes process steps to extract a feature vector from the luminance component of input pixel window 31 , and select a key vector of a representative class from a plurality of class representative vectors with corresponding filter coefficients, closest to the feature vector. The process for determining the set of key vectors with corresponding filter coefficients is described in applicants patent application Ser. No. 09/064,638 filed Apr. 21, 1998, titled “APPARATUS AND METHOD OF BUILDING AN ELECTRONIC DATABASE FOR RESOLUTION SYNTHESIS”, incorporated herein by reference.

More particularly, in Step S 701 , compute luminance for pixel window 31 as described in step S 402 of FIG. 4 .

In Step S 702 , acquire luminance feature vector Y as described in applicant's patent application Ser. No. 09/064,638 filed Apr. 21, 1998, titled “APPARATUS AND METHOD OF BUILDING AN ELECTRONIC DATABASE FOR RESOLUTION SYNTHESIS”, incorporated herein by reference. Other vectors using other features for describing a color image, for example, hue and saturation etc., may also be used to implement this aspect of the present invention.

In Step S 703 assign a maximum Euclidean distance (D m ) for comparing a class representative vector to feature vector Y.

In Step S 704 , acquire a first class representative vector from a plurality of class representative vectors.

In Step S 705 , calculate the squared Euclidean distance D 1 between the first class representative vector and feature vector Y, using X number of N Terms of the first class representative vector.

In Step S 706 , compare D 1 with D m and determine if D 1 is greater than D m . If D 1 is greater than D m , the process moves to stop S 712 .

If D 1 is less than D m , then in Step S 707 , calculate Squared Euclidean distance D 2 between the first class representative vector and feature vector Y, using N terms of the first class representative.

In Step S 708 , compare D 2 with a predetermined threshold distance D th . If D 2 is less than D th , then the process moves to Step S 713 .

If D 2 is greater than D th , then in Step S 709 , determine if D 2 is less than D m .

If D 2 is less than D m , then in step S 710 , reset D m as D 2 .

In Step S 711 , assign the first class where D 2 is less than D m , as the current best class.

If D 2 is greater than D m , then the process moves to step S 712 .

In Step S 712 , determine if all classes have been analyzed. If all classes are analyzed then the process flow moves to Step S 713 , where filter coefficients of the current best class are selected. If all the classes have not been analyzed, process moves back to Step S 704 to analyze the next class representative vector, for example, the second class representative vector.

Although the present invention has been described with reference to specific embodiments, these embodiments are illustrative only and not limiting. Many other applications and embodiments of the present invention will be apparent in light of this disclosure and the following claims.