Image scaler with controllable sharpness

1. Field of the Invention

The present invention relates to digital image and video processing. More specifically, the present invention relates to methods of scaling images with controllable sharpness to reduce blurring, grain effects, and saw tooth effects.

2. Discussion of Related Art

Due to advancing semiconductor processing technology, integrated circuits (ICs) have greatly increased in functionality and complexity. With increasing processing and memory capabilities, many formerly analog tasks are being performed digitally. For example, many digital display systems, such as Liquid Crystal Displays (LCDs), micro mirror systems, and plasma displays, are being used in place of analog television sets. These digital display systems have a set number of pixels, i.e. the native display resolution. For example, large screen LCDs may have a native resolution as high as 1920×1080 pixels. However small portable digital display system may have native display resolution as low as 320×200 pixels.

Various video formats have different resolutions. For example, NTSC resolution is 720×480, PAL resolution is 720×576, HDTV standards include 1920×1080, 1280×720, and 640×480, etc. To correctly display a video signal, the digital display system must resize the images of the video signal for the native display resolution. For example as illustrated in FIG. 1 , an image 110 is scaled up (i.e. enlarged) for a large display 120 . Conversely as illustrated in FIG. 2 , image 110 is scaled down (i.e. reduced) for a small display 220 .

In the scaling process each pixel of the scaled image is generated by determining a corresponding position in the original image and generating values for the pixel of the scaled image based on the pixels of the original image near the corresponding position. For example as illustrated in FIG. 3 , pixel 321 in scaled image 320 corresponds to a corresponding position 321 b that is located between pixels 311 , 312 , 313 , and 314 of original image 310 . Some conventional scalers calculate pixel values, e.g. luminance, U chrominance, and V chrominance values in YUV format, for pixel 321 by copying the corresponding values of the pixel in original image 310 that is nearest to corresponding position 321 b . Other conventional scalers use bi-linear interpolation based on the values of the 4 pixels nearest corresponding position 321 b , i.e., pixels 311 , 312 , 313 , and 314 . Other pixel formats could also be used such as RGB. For clarity, the present invention is described using YUV format, however one skilled in the art can easily adapt the teachings presented herein for other pixel formats.

The corresponding position to a pixel of the scaled image can be calculated using the location of the pixel in the scaled image and a horizontal scaling factor HSF and a vertical scaling factor VSF. The scaling factors are based on the relative dimensions of original image to the scaled image. For clarity, the examples described herein use an original image having I pixels in each row and J pixels in each column. The scaled image has X pixels in each row and Y pixels in each column. A pixel P(x, y) in the scaled image is the pixel in the x-th column, and y-th row of the scaled image, where pixel P( 0 , 0 ) is the top left pixel and pixel P(X−1, Y−1) is the bottom right pixel of the scaled image. Horizontal scaling factor HSF is equal to the horizontal dimension of the original image (in pixels) minus one divided by the horizontal dimension of the scaled image (in pixels) minus one, i.e. HSF=(I−1)/(X−1). However, many systems simplify the calculation of horizontal scaling factor HSF by using the horizontal dimension of the original image (in pixels) divided by the horizontal dimension of the scaled image, i.e. HSF=I/X. Similarly, vertical scaling factor VSF is equal to the vertical dimension of the original image (in pixels) minus one divided by the vertical dimension of the scaled image (in pixels) minus one, i.e. VSF=(J−1)/(Y−1). However, many systems simplify the calculation of vertical scaling factor VSF by using the vertical dimension of the original image (in pixels) divided by the vertical dimension of the scaled image, i.e. VSF=J/Y.

The corresponding position of pixel P(x, y) is defined with a horizontal position HP(x) and a vertical position VP(y). Horizontal position HP(x) is equal to x multiplied by horizontal scaling factor HSF (i.e., HP(x)=x*HSF). Vertical position VP(y) is equal to y multiplied by vertical scaling factor VSF (i.e., VP(y)=y*VSF).

As digital displays become larger, flaws of conventional scalers become magnified. For example, images that are scaled up (enlarged) become blurred because individual pixel values from the original image are combined to form the scaled image. Thus, the effect of scaling up an image has the effect of applying a low-pass filter to the image, which reduces the sharpness of the scaled image as compared to the original image. In zero-th order scalers, blurring is not as prevalent, however blockiness in the scaled image becomes a problem. The blockiness is caused by copying a single pixel of the original image to multiple adjacent pixels in the scaled image.

Another flaw of conventional scalers is saw tooth artifacts along diagonal lines in a scaled-up image. Theoretically, saw tooth artifacts are present along diagonal lines in all digital images due to finite resolution. Normally, the saw tooth artifacts are not visible in high-resolution images. However, when an image is scaled up, the values of the pixels in the original image are used to calculate multiple pixels in the scaled image, which may enlarge the saw tooth artifacts.

Scaling down of images also produces flaws in the scaled image. For examples scaling down (i.e. reducing) an image may produce a scaled image that is grainy. Specifically, scaling down has the effect of applying a high-pass filter to an image, which would emphasize rapid transitions within an image, which results in a grainy image.

Hence, there is a need for a method or system that can efficiently scale an image without the flaws of conventional scalers that may produce saw toothed, blurry, blocky, or grainy scaled images.

Accordingly, the present invention provides a method and system for scaling an image that uses a high frequency adjusted position rather than the corresponding position of a current pixel during scaling. The high frequency adjusted position is calculated based on the high frequency components of the pixels near the corresponding position. By accounting for the high frequency components of nearby pixels, image scalers according to the present invention produce scaled images of higher quality than conventional scalers. Furthermore, some embodiments of the present invention also use an adjustable sharpness compensation to further reduce flaws that are common to conventional scalers.

In accordance with the present invention an image scaler, configured to produce a scaled image from an original image, calculates a corresponding position in the original image for a current pixel of the scaled image. The corresponding position includes a horizontal position and a vertical position. The image scaler also calculates a high frequency adjusted position, which includes a high frequency adjusted horizontal position and a high frequency adjusted vertical position, in the original image for the current pixel. Pixel values for the current pixel are generated by the image scaler using the high frequency adjusted position rather than the corresponding position. Generally, the high frequency adjusted position is moved horizontally in a horizontal direction of larger horizontal high frequency components and is moved vertically in a vertical direction of larger vertical high frequency components.

Some embodiments of the present invention also use sharpness compensation to improve the quality of the scaled image. In these embodiments, the sharpness compensation is proportional to an interpolation at the high frequency adjusted position of high frequency components of pixels near the corresponding position. For example, one specific embodiment of the present invention calculates the sharpness compensation using a bilinear interpolation at the high frequency adjusted position of the high frequency components of the four pixels nearest the corresponding position. The sharpness compensation is added to the luminance value of the current pixel.

The present invention will be more fully understood in view of the following description and drawings.

FIG. 1 is an illustration of up-scaling an image.

FIG. 2 is an illustration of down-scaling an image.

FIG. 3 illustrates the relationship between a pixel of a scaled image and the corresponding position of the pixel in an original image.

FIG. 4 is a block diagram of one embodiment of the present invention.

FIG. 5 illustrates the relationship of a high frequency adjusted position and a corresponding position in accordance with one embodiment of the present invention.

FIG. 6 illustrates the pixels surrounding a corresponding position and a high frequency adjusted position in accordance with one embodiment of the present invention.

As explained above, conventional scaling techniques may cause blurring, blocking, and saw tooth artifacts in an up-scaled (i.e., enlarged) image and may cause excessive graininess in a down-scaled (i.e., reduced) image. To reduce saw tooth artifacts, the present invention uses a high frequency adjusted position in place of the corresponding position to generate a pixel P(x, y) of the scaled image. Specifically, the high frequency adjusted position is moved horizontally in the direction of the larger magnitude of horizontal high frequency components. Similarly the high frequency adjusted position is moved vertically in the direction of the larger magnitude of vertical high frequency components. The amount of movement is proportional to the high-frequency components of the neighboring pixels. Furthermore, the present invention adjusts the sharpness of the scaled image to reduce the flaws of the scaled image. Specifically, the present invention calculates a sharpness compensation value, which is used in generating the pixels of the scaled image.

FIG. 4 is a block diagram of an image scaler 400 in accordance with one embodiment of the present invention. Image scaler 400 includes a corresponding position calculation unit 410 , an adder 420 , a scaling unit 430 , an adder 440 , a video buffer 450 , a local high frequency components calculation unit (LHFCCU) 460 , a position adjustment unit 470 , and a sharpness compensation unit 480 . Image scaler 400 generates the pixels of the scaled image using data from the original image that is stored in video buffer 450 . To minimize the size of video buffer 450 , the scaled image is generated line by line (and pixel by pixel within each line) so that only a few lines of pixels are stored in video buffer 450 . In general a line of pixels corresponds to a line buffer in video buffer 450 . The number of lines of pixels stored in video buffer 450 depends on the scaling technique used in scaling unit 430 and the high frequency components calculation techniques used in local high frequency components calculation unit 460 . Video buffer 450 is typically implemented as a circular buffer, which maintains the number of lines of pixel data that are necessary for the computations used in the scaling techniques and local high frequency components calculation. However, extra line buffers may be needed for pre-loading of future data. Thus, for example an embodiment of the present invention that requires four lines of pixels for computations may have a video buffer that includes more than four line buffers.

To generate a pixel P(x, y) of the scaled image, corresponding position calculation unit 410 calculates horizontal position HP(x) and vertical position VP(y) of the corresponding position CP (see FIG. 5 ) of pixel P(x, y), as described above. Adder 420 adds a horizontal high frequency position adjustment HHFPA to horizontal position HP(x) to generate a high-frequency adjusted horizontal position HFAHP. Adder 420 also adds a vertical high-frequency position adjustment VHFPA to vertical position VP(y) to generate a high-frequency adjusted vertical position HFAVP. Horizontal high frequency position adjustment HHFPA and vertical high frequency position adjustment VHFPA are generated by position adjustment unit 470 as described below. High frequency adjusted horizontal position HFAHP and high frequency adjusted vertical position HFAVP are the coordinates of high frequency adjusted position HFAP (see FIG. 5 ).

FIG. 5 illustrates the relationship between corresponding position CP and high frequency adjusted position HFAP. Specifically, corresponding position CP is located at horizontal position HP(x) and vertical position VP(y) while high frequency adjusted position HFAP is located at high frequency adjusted horizontal position HFAHP and high frequency adjusted vertical position HFAVP. High frequency adjusted horizontal position HFAHP is equal to horizontal position HP(x) plus horizontal high frequency position adjustment HHFPA (i.e., HFAHP=HP(x)+HHFPA). High frequency adjusted vertical position HFAVP is equal to vertical position VP(x) plus vertical high frequency position adjustment VHFPA (i.e., HFAVP=VP(y)+VHFPA).

Returning to FIG. 4 , scaling unit 430 generates an intermediate pixel IP(x, y) using high frequency adjusted horizontal position HFAHP, high frequency adjusted vertical position HFAVP, and the pixels of the original image stored in video buffer 450 . The specific scaling technique used by scaling unit 430 is not an integral part of the present invention. For example, some embodiments of the present invention use bi-linear scaling, other embodiments use zero-th order scaling, and still other embodiments may use bi-cubic, splines or other well-known scaling techniques. In most embodiments of the present invention, scaling unit 430 uses high frequency adjusted horizontal position HFAHP and high frequency adjusted vertical position HFAVP for generating both luminance and chrominance values for intermediate pixel IP(x, y). However, some embodiments of the present invention use horizontal position HP(x) and vertical position VP(y) for chrominance values.

Adder 440 adds a sharpness compensation S_C to the luminance portion of the intermediate pixel value IP(x, y) to generate the luminance portion of pixel P(x, y) for the scaled image. Sharpness compensation S_C is generated by sharpness compensation unit 480 as described below. The chrominance portions of intermediate pixel value IP(x, y) are not affected by sharpness compensation S_C.

Local high frequency components calculation unit 460 calculates high frequency components for the pixels neighboring corresponding position CP. For clarity the nomenclature for the pixels neighboring corresponding position CP are illustrated in FIG. 6 . Specifically, the four pixels closest to corresponding position CP are referenced as P 6 , P 7 , P 10 , and P 11 , where pixel P 6 is above and to the left of corresponding position CP, pixel P 7 is above and to the right of corresponding position CP, pixel P 10 is below and to the left of corresponding position CP, and pixel P 11 is below and to the right of corresponding position CP. Furthermore, the next twelve closest pixels to corresponding position CP are referenced with P 1 , P 2 , P 3 , P 4 , P 5 , P 8 , P 9 , P 12 , P 13 , P 14 , P 15 , and P 16 . Therefore, the 16 closest pixels to corresponding position CP, which are laid out in 4 rows of 4 pixels, are referenced sequentially from left to right top to bottom as illustrated in FIG. 6 . Corresponding position CP is offset from pixel P 6 by a horizontal offset H_O and a vertical offset V_O. If corresponding position CP is exactly on a pixel, then that pixel is used as pixel P 6 . Thus, horizontal offset H_O and vertical offset V_O would both be equal to zero.

Local high frequency component calculation unit 460 calculates the local high frequency component for each of the four closest pixels to corresponding position CP, i.e. for pixels P 6 , P 7 , P 10 , and P 11 . Various techniques can be used to calculate the local high frequency components. Specifically, the local high frequency component at a pixel is equal to a high pass filter result at the pixel. Thus, any high pass filter, such as Laplacian Operator or difference of Gaussians, could be used to calculate local high frequency components. In some embodiments of the present invention, local high frequency components calculation unit 460 calculates both a horizontal high frequency component and a vertical high frequency component for pixels P 6 , P 7 , P 10 , and P 11 . In a particular embodiment of the present invention, the horizontal high frequency component of a specific pixel is equal to two times luminance of the specific pixel minus the luminance of the pixel to the left of the specific pixel minus the luminance of the pixel to the right of the specific pixel. For example, the horizontal high frequency component for pixel P 6 (i.e., HHFC 6 ) is equal to the two times the luminance of pixel P 6 minus the luminance of pixel P 5 minus the luminance of pixel P 7 . For clarity the luminance of a pixel Pt is referenced as luminance Yt. For example, the luminance of pixel p 6 is referenced as luminance Y 6 . Equations EQ1a, EQ2a, EQ3a, and EQ4a show symbolically how to calculate horizontal high frequency components HHFC 6 , HHFC 7 , HHFC 10 , and HHFC 11 , for pixels P 6 , P 7 , P 10 , and P 11 , respectively, for this embodiment of the present invention.

HHFC 6=2* Y 6− Y 5− Y 7  (EQ1a)

HHFC 7=2* Y 7− Y 6− Y 8  (EQ2a)

HHFC 10=2* Y 10− Y 9− Y 11  (EQ3a)

HHFC 11=2* Y 11− Y 10− Y 12  (EQ4a)

For the embodiment of the present invention using equations EQ1a, EQ2a, EQ3a, and EQ4a, the vertical high frequency component of a specific pixel is equal to two times luminance of the specific pixel minus the luminance of the pixel above the specific pixel minus the luminance of the pixel below the specific pixel. For example, the vertical high frequency component of pixel P 6 (i.e., VHFC 6 ) is equal to two times the luminance of pixel P 6 minus the luminance of pixel P 2 minus the luminance of pixel P 10 . Equations EQ1b, EQ2b, EQ3b, and EQ4b shows symbolically how to calculate vertical high frequency components VHFC 6 , VHFC 7 , VHFC 10 , and VHFC 11 , for pixels P 6 , P 7 , P 10 , and P 11 , respectively

VHFC 6=2* Y 6− Y 2− Y 10  (EQ1b)

VHFC 7=2* Y 7− Y 3− Y 11  (EQ2b)

VHFC 10=2* Y 10− Y 6− Y 14  (EQ3b)

VHFC 11=2* Y 11− Y 7− Y 15  (EQ4b)

As explained above, local high frequency components can be calculated in a variety of techniques. The equation EQ1a, EQ1b, EQ2a, EQ2b, EQ3a, EQ3b, EQ4a, and EQ4b are for one particular embodiment of the present invention. One skilled in the art can use the principles of the present invention with other techniques to calculate local high frequency components.

At or near the edges of an image, local high frequency components cannot be defined for every direction. For example, at the bottom of the image, pixels P 13 , P 14 , P 15 , and P 16 are not available for calculating the local high frequency components. For most embodiments of the present invention, when the local high frequency components can not be calculated for a pixel of the scaled image, high frequency adjusted position HFAP remains with corresponding position CP.

The high frequency components (HFCs) generated by local high frequency components calculation unit 460 are provided to position adjustment unit 470 and sharpness compensation unit 480 . Position adjustment unit 470 calculates horizontal high frequency position adjustment HHFPA and vertical high frequency position adjustment VHFPA. As explained above, high frequency adjusted position HFAP should be moved horizontally in the direction of the larger magnitude of horizontal high frequency components and vertically in the direction of the larger magnitude of vertical high frequency components. Therefore, position adjustment unit 470 calculates horizontal high frequency position adjustment HHFPA and vertical high frequency position adjustment VHFPA.

To calculate horizontal high frequency position adjustment HHFPA, position adjustment unit 470 calculates a left high frequency component LHFC, which is equal to a weighted sum of the horizontal high frequency components of the two closest pixels to the left of current position CP, i.e. pixels P 6 and P 10 . Specifically, left high frequency component LHFC is equal to the product of horizontal high frequency component HHFC6 of pixel P 6 with the difference between one and vertical offset V_O plus the product of horizontal high frequency component HHFC 10 of pixel P 10 with vertical offset V_O. Equation EQ5 shows symbolically how to calculate left high frequency component LHFC.

LHFC=HHFC 6*(1− V — O )+ HHFC 10* V — O   (EQ5)

Similarly, position adjustment unit 470 calculates a right high frequency component RHFC, which is equal to a weighted sum of the horizontal high frequency components of the two closest pixels to the right of current position CP, i.e. pixels P 7 and P 11 . Specifically, right high frequency component RHFC is equal to the product of horizontal high frequency component HHFC 7 of pixel P 7 with the difference between one and vertical offset V_O plus the product of horizontal high frequency component HHFC 11 of pixel P 11 with vertical offset V_O. Equation EQ6 shows symbolically how to calculate right high frequency component RHFC.

RHFC=HHFC 7*(1− V — O )+ HHFC 11* V — O   (EQ6)

As explained above high frequency adjusted position HFAP is moved horizontally in the direction of the larger magnitude of horizontal high-frequency component. The magnitude of a horizontal high-frequency component is equal to the absolute value of the horizontal high frequency component. Thus, when the absolute value of right high frequency component RHFC is greater than the absolute value of left high frequency component LHFC then the high frequency adjusted position HFAP should be moved to the right of corresponding position CP. Conversely, when the absolute value of right high frequency component RHFC is less than the absolute value of left high frequency component LHFC then the high frequency adjusted position HFAP should be moved to the left of corresponding position CP. However, when the absolute value of right high frequency component RHFC is equal to the absolute value of left high frequency component LHFC then the high frequency adjusted position HFAP should not be moved horizontally from corresponding position CP and horizontal high frequency position adjustment HHFPA should be equal to zero.

The amount of horizontal high frequency position adjustment is in proportion to the absolute difference between absolute left-side and absolute right-side high frequency components. Therefore, position adjustment 470 calculates a horizontal high frequency difference HHFD, which is equal to the absolute value of the difference between the absolute value of right high frequency component RHFC and the absolute value of left high frequency component LFHC. Equation EQ7 shows symbolically how to calculate horizontal high frequency difference HHFD.

HHFD=||RHFC|−|LHFC||   (EQ7)

To provide greater user control, two user configurable parameters are used in the calculation of horizontal high frequency position adjustment HHFPA. The first user configurable parameter is a high frequency difference threshold HFDT, which limits the amplitude of horizontal high frequency difference HHFD. Specifically, if horizontal high frequency difference HHFD is greater than high frequency difference threshold, then horizontal high frequency difference HHFD is reset to be equal to high frequency difference threshold. Because high frequency difference threshold HFDT is used as a divisor (as explained below) using a power of 2 for high frequency difference threshold would simplify the circuits required for calculating horizontal high frequency position adjustment HHFPA. Alternatively, a register configured with the reciprocal of high frequency difference threshold HFDT (i.e., 1/HFDT) could be used to avoid using high frequency difference threshold HFDT as a divisor. Specifically, instead of dividing by high frequency difference threshold HFDT multiplying by the reciprocal of high frequency difference threshold HFDT is performed. In one embodiment of the present invention high frequency difference threshold HFDT has a default value of 16.

The second user configurable parameter is an interpolation point adjustment parameter IPAP, which has a range of 0 to 1, inclusive. IPAP is used to further control the adjustment of high frequency adjusted position HFAP. The amount of adjustment is proportional to interpolation point adjustment parameter IPAP. When interpolation point adjustment parameter IPAP is equal to zero, adjustment of high frequency adjusted position HFAP is eliminated. Generally, the default value of interpolation point adjustment parameter IPAP is equal to 0.5

The magnitude of horizontal position adjustment MHPA is equal to horizontal scaling factor HSF multiplied by interpolation adjustment parameter IPAP, multiplied by horizontal high frequency difference HHFD divided by high frequency difference threshold HFDT. As explained above, dividing by high frequency difference threshold HFDT can be avoided by multiplying with the reciprocal of high frequency difference threshold HFDT. In equation EQ8, horizontal scaling factor HSF is equivalent to a “horizontal step” between two pixels of the scaled image. Equation EQ8 shows symbolically how to calculate magnitude of horizontal position adjustment MHPA.

MHPA=HSF*IPAP *( HHFD/HFDT )  (EQ8)

When the absolute value of right high frequency component RHFC is greater than the absolute value of left high frequency component LHFC then the high frequency adjusted position HFAP should be moved to the right of corresponding position CP. However in most embodiments of the present invention, high frequency adjusted position HFAP should not move beyond pixels P 6 , P 7 , P 10 , and P 11 because additional pixels would be needed to for interpolation. For example, if high frequency adjusted position HFAP is moved below P 10 and P 11 , the pixels below pixels P 14 and P 15 would be needed for interpolation and extra line buffers would be required in the video buffer. Therefore, when the absolute value of right high frequency component RHFC is greater than the absolute value of left high frequency component LHFC and one minus horizontal offset H_O is greater than or equal to magnitude of horizontal position adjustment MHPA then horizontal high frequency position adjustment HHFPA is equal to magnitude of horizontal position adjustment MHPA. However, when the absolute value of right high frequency component RHFC is greater than the absolute value of left high frequency component LHFC and one minus horizontal offset H_O is less than magnitude of horizontal position adjustment MHPA then horizontal high frequency position adjustment HHFPA is equal to one minus horizontal offset H_O.

Conversely, when the absolute value of right high frequency component RHFC is less than the absolute value of left high frequency component LHFC then the high frequency adjusted position HFAP should be moved to the left of corresponding position CP. However, high frequency adjusted position HFAP can not move beyond pixels P 6 and P 10 in the horizontal direction. Therefore, when the absolute value of right high frequency component RHFC is less than the absolute value of left high frequency component LHFC and horizontal offset H_O is greater than magnitude of horizontal position adjustment MHPA then horizontal high frequency position adjustment HHFPA is equal to magnitude of horizontal position adjustment MHPA multiplied by negative one. However, if horizontal offset H_O is less than magnitude of horizontal position adjustment MHPA, horizontal high frequency position adjustment HHFPA is equal to horizontal offset H_O multiplied by negative one.

When the absolute value of right high frequency component RHFC is equal to the absolute value of left high frequency component LHFC then the high frequency adjusted position HFAP should not be moved from corresponding position CP and horizontal high frequency adjustment HHFPA should be equal to zero.

Table 1 provides a pseudo code listing which shows symbolically how to calculate horizontal high frequency position adjustment HHFPA.

To calculate vertical high frequency position adjustment VHFPA, position adjustment unit 470 calculates a top high frequency component THFC, which is equal to a weighted sum of the vertical high frequency components of the two closest pixels above of current position CP, i.e. pixels P 6 and P 7 . Specifically, top high frequency component THFC is equal to the product of vertical high frequency component VHFC 6 of pixel P 6 with the difference between one and horizontal offset H_O plus the product of vertical high frequency component VHFC 7 of pixel P 7 with horizontal offset H_O. Equation EQ9 shows symbolically how to calculate top high frequency component THFC.

THFC=VHFC 6*(1− H — O )+ VHFC 7* H — O   (EQ9)

Similarly, position adjustment unit 470 calculates a bottom high frequency component BHFC, which is equal to a weighted sum of the vertical high frequency components of the two closest pixels below current position CP, i.e. pixels P 10 and P 11 . Specifically, bottom high frequency component BHFC is equal to the product of vertical high frequency component VHFC 10 of pixel P 10 with the difference between one and horizontal offset H_O plus the product of vertical high frequency component VHFC 11 of pixel P 11 with horizontal offset H_O. Equation EQ10 shows symbolically how to calculate bottom high frequency component BHFC.

BHFC=VHFC 10*(1− H — O )+ VHFC 11* H — O   (EQ10)

As explained above high frequency adjusted position HFAP is moved vertically in the direction of the larger magnitude of vertical high-frequency component. The magnitude of a vertical high-frequency component is equal to the absolute value of the vertical high frequency component. Thus, when the absolute value of bottom high frequency component BHFC is greater than the absolute value of top high frequency component THFC then the high frequency adjusted position HFAP should be moved below corresponding position CP. Conversely, when the absolute value of bottom high frequency component BHFC is less than the absolute value of top high frequency component THFC then the high frequency adjusted position HFAP should be moved above corresponding position CP. However, when the absolute value of bottom high frequency component BHFC is equal to the absolute value of top high frequency component THFC then the high frequency adjusted position HFAP should not be moved vertically from corresponding position CP and vertical high frequency adjustment VHFPA should be equal to zero.

The amount of high frequency position adjustment is in proportion to the absolute difference between absolute top high frequency components and absolute bottom high frequency components. Therefore, position adjustment unit 470 calculates a vertical high frequency difference VHFD, which is equal to the absolute value of the difference between the absolute value of bottom high frequency component BHFC and the absolute value of top high frequency component TFHC. Equation EQ11 shows symbolically how to calculate vertical high frequency difference VHFD.

VHFD=||BHFC|−|THFC||   (EQ11)

As with horizontal high frequency position adjustment HHFPA, high frequency difference threshold HFDT is also used to limit the amplitude of vertical high frequency difference VHFD. Specifically, if vertical high frequency difference VHFD is greater than high frequency difference threshold, then vertical high frequency difference VHFD is reset to be equal to high frequency difference threshold. Furthermore, interpolation point adjustment parameter IPAP, is also used to control the vertical adjustment of high frequency adjusted position HFAP.

The magnitude of vertical position adjustment MVPA is equal to vertical scaling factor VSF multiplied by interpolation adjustment parameter IPAP, multiplied by vertical high frequency difference VHFD divided by high frequency difference threshold HFDT. As explained above, dividing by high frequency difference threshold HFDT can be avoided by multiplying with the reciprocal of high frequency difference threshold HFDT. Equation EQ12 shows symbolically how to calculate magnitude of vertical position adjustment MVPA. In equation EQ12, vertical scaling factor VSF is equivalent to a “vertical step” between two pixels of the scaled image.

MVPA=VSF*IPAP *( VHFD/HFDT )  (EQ12)

When the absolute value of bottom high frequency component BHFC is greater than the absolute value of top high frequency component THFC then the high frequency adjusted position HFAP should be moved below corresponding position CP. However, as explained above, in most embodiments of the present invention, high frequency adjusted position HFAP should not move beyond pixels P 6 , P 7 , P 10 , and P 11 because additional pixels would be needed to for interpolation. Therefore, when the absolute value of bottom high frequency component BHFC is greater than the absolute value of top high frequency component THFC and one minus vertical offset V_O is greater than or equal to magnitude of vertical position adjustment MVPA then vertical high frequency position adjustment is equal to magnitude of vertical position adjustment MVPA. However, when the absolute value of bottom high frequency component BHFC is greater than the absolute value of top high frequency component THFC and one minus vertical offset V_O is less than magnitude of vertical position adjustment MVPA then vertical high frequency position adjustment is equal to one minus vertical offset V_O.

Conversely, when the absolute value of bottom high frequency component BHFC is less than the absolute value of top high frequency component THFC then the high frequency adjusted position HFAP should be moved above corresponding position CP. However, high frequency adjusted position HFAP should not move beyond pixels P 6 and P 7 in the vertical direction. Therefore, when the absolute value of bottom high frequency component BHFC is less than the absolute value of top high frequency component THFC and vertical offset V_O is greater than magnitude of vertical position adjustment MVPA then vertical high frequency position adjustment is equal to magnitude of vertical position adjustment MVPA multiplied by negative one. However, if vertical offset V_O is less than magnitude of vertical position adjustment MVPA, vertical high frequency position adjustment VHFPA is equal to vertical offset V_O multiplied by negative one.

When the absolute value of bottom high frequency component BHFC is equal to the absolute value of top high frequency component THFC then the high frequency adjusted position HFAP should not be moved from corresponding position CP and vertical high frequency adjustment VHFPA should be equal to zero.

Table 2 provides a pseudo code listing which shows symbolically how to calculate vertical high frequency position adjustment VHFPA.

Returning to FIG. 4 , sharpness compensation unit 480 , which receives the various high frequency components from local high frequency components calculation unit 460 , horizontal high frequency position adjustment HHFPA and vertical high frequency position adjustment VHFPA from position adjustment unit 470 and a user configurable sharpness control parameter SCP, generates sharpness compensation S_C for adder 440 . Specifically, sharpness compensation unit 480 generates sharpness compensation S_C to be equal to sharpness control parameter SCP multiplied by an interpolation at high frequency adjusted position HFAP of the high-frequency components of pixels P 6 , P 7 , P 10 , and P 11 . For example, in one embodiment of the present invention, a zero-th order interpolation is used so that sharpness compensation S_C is equal to sharpness control parameter SCP multiplied by the sum of high frequency components of the pixel closest to high frequency adjusted position HFAP. Another embodiment of the present invention uses a bilinear interpolation, i.e. sharpness compensation S_C to be equal to sharpness control parameter SCP multiplied by a bilinear interpolation at high frequency adjusted position HFAP of the high-frequency components of pixels P 6 , P 7 , P 10 , and P 11 . Equation EQ13 shows symbolically how to calculate sharpness compensation S_C using a bilinear interpolation. Higher order interpolation techniques could also be used. However, the high frequency components of additional pixels may be necessary for other interpolation techniques.

Sharpness control parameter SCP can be any real number. However to reduce the complexity of sharpness compensation unit 480 , sharpness control parameter is usually limited to a smaller range such as negative 8 to positive 8, inclusive. Positive values of sharpness control parameter SCP enhance the sharpness of the resulting image while negative values of sharpness control parameter SCP reduce the sharpness of the resulting image. Therefore, when image scaler 400 is up-scaling in image (i.e. enlarging an image), sharpness control parameter SCP should be positive to enhance sharpness, which would reduce the blurring problem described above. However, if image scaler 400 is down-scaling an image (i.e. reducing an image) sharpness control parameter SCP should be negative to reduce sharpness, which would reduce the graininess problem described above. To switch off sharpness compensation, sharpness control parameter SCP should be set to zero, which would force sharpness compensation S_C to be equal to zero.

In some embodiments of the present invention, sharpness compensation unit 480 includes a sharpness compensation threshold SCT, which limits the magnitude of sharpness compensation S_C. Specifically, if sharpness compensation S_C is greater than sharpness compensation threshold SCT, sharpness compensation S_C is set equal to sharpness compensation threshold SCT. Conversely, if sharpness compensation S_C is less than sharpness compensation threshold SCT multiplied by negative one, sharpness compensation S_C is set equal to sharpness compensation threshold SCT multiplied by negative one. Some embodiments of the present invention may include a sharpness compensation coring threshold SCCT. When sharpness compensation S_C is larger than negative one multiplied by sharpness compensation coring threshold SCCT and smaller than sharpness compensation coring threshold SCCT, sharpness compensation S_C is set equal to zero. Sharpness compensation coring threshold SCCT is used to avoid affecting of small changes in pixel luminance caused by noise.

In the various embodiments of the present invention, novel structures and methods have been described for scaling images. The various embodiments of the structures and methods of this invention that are described above are illustrative only of the principles of this invention and are not intended to limit the scope of the invention to the particular embodiments described. For example, in view of this disclosure those skilled in the art can define other scaling techniques, high frequency components, scaling units, position adjustment units, sharpness compensation units, video buffers, and so forth, and use these alternative features to create a method, circuit, or system according to the principles of this invention. Thus, the invention is limited only by the following claims.