Direction-adaptive image upsampling system and method using double interpolation

1. Field of the Invention

The present invention generally relates to image interpolation, and more particularly to a direction-adaptive image upsampling system and method using double interpolation.

2. Description of Related Art

Image upsampling, or image interpolation, is a process of increasing the resolution of an image. It has become an essential technique in many applications, such as image editing software, TV scaler and surveillance systems, and particularly in real-time applications.

Image upsampling may be performed either based on multiple input images or on a single input image. The former one, i.e., the multi-image upsampling, needs long computation time and is thus not suitable for real-time applications.

The latter one, i.e., the single-image upsampling, may be generally classified into three categories: the interpolation-based scheme, the statistic-based scheme and the learning-based scheme. Bilinear and bicubic are commonly used interpolation-based methods, but usually produce zigzagging artifact on the edges. Although further methods have been proposed to solve the zigzagging artifacts, time-consuming edge detection, however, is required. The statistic-based scheme employs edge statistics to reconstruct high-resolution images, but requires several iterations to arrive at a good result. The learning-based scheme predicts high-frequency details by database that requires high computation time.

For the foregoing reasons, a need has arisen to propose a novel interpolation-based scheme that has a simple and robust algorithm with high image quality suitable for real-time applications.

In view of the foregoing, it is an object of the embodiment of the present invention to provide a direction-adaptive image upsampling system and method using double interpolation such that the edge can be interpolated along the right direction without performing any time-consuming edge detection.

According to one embodiment, the direction-adaptive image upsampling system using double interpolation includes a plurality of double interpolation (DI) units with different interpolation functions, a decision unit and an adaptive selector. Each DI unit is coupled to receive an input image and generate a corresponding double-interpolated image and an associated double-interpolated difference, wherein, for each pixel and each said DI unit, the double-interpolated difference is a pixel value difference between the input image and the associated double-interpolated image. The decision unit is coupled to receive the double-interpolated differences from the plurality of DI units for deciding the DI unit that has an optimal interpolation around a pixel under process. The adaptive selector, for each pixel, is configured to select the double-interpolated image associated with the DI unit having the optimal interpolation as an output image.

FIG. 1 shows a block diagram illustrative of a direction-adaptive image upsampling system and method using double interpolation according to one embodiment of the present invention;

FIG. 2 shows a detailed block diagram of the DI unit of FIG. 1 ; and

FIG. 3 shows exemplary pixels of an interpolated image with the upsampling factor of 2.

FIG. 1 shows a block diagram illustrative of a direction-adaptive image upsampling system and method using double interpolation according to one embodiment of the present invention.

In the embodiment, an input image is first interpolated by several double interpolating (DI) units 10 with different interpolation functions H I (e.g., H 0 , H 1 , etc.). For example, a preferred embodiment adopts five interpolation functions, which includes a bicubic interpolation and four bilinear interpolations with directions of about 27°, −27°, 63° and −63°, respectively. Each DI unit 10 accordingly generates a corresponding double-interpolated image G I (e.g., G 0 , G 1 , etc.) and an associated double-interpolated difference D I (e.g., D 0 , D 1 , etc.). For each pixel, the double-interpolated difference D I is a pixel value difference between the input image and the double-interpolated image G I .

Subsequently, for each pixel, a decision unit 12 is coupled to receive the double-interpolated differences D I : in order to decide the DI unit 10 (or its associated interpolation function) that has an optimal interpolation around the pixel under process. In the embodiment, with respect to each DI unit 10 with the interpolation function H I , a cost function C I is calculated in the decision unit 12 by summing the associated double-interpolated differences D I of the pixel (x,y) under process and its neighboring pixels:

C I ⁡ ( x , y ) = ∑ u , v ∈ N ⁢ ⁢ D I ⁡ ( x + u , y + v )

where (u,v) belongs to a neighbored window N (e.g., a 7×7 (pixels) window) made of the neighboring pixels.

Among the calculated cost functions C I with respect to the interpolation functions H I of the DI units 10 , the cost function C I with the least value (i.e., the minimum cost function) is determined, and the associated interpolation function H I is then determined as the interpolation function that has an optimal interpolation around the pixel under process.

Finally, for each pixel, an adaptive selector 14 is coupled to receive the double-interpolated images G I from the DI units 10 , and the double-interpolated image G I corresponding to the minimum cost function C I is selected as an output image. According to the embodiment, by employing the double-interpolated difference as a quality measurement of an interpolation algorithm and locally choosing the minimum difference, the best prediction of pixel value among different interpolation algorithms can be chosen and the advantages of all the interpolation functions can be combined. That is, the edge can be interpolated along the right direction without performing any edge detection or calculating the orientation of gradient. Therefore, the embodiment can provide an artifact-free output in short computation time and can thus be suitable for real time applications.

FIG. 2 shows a detailed block diagram of the DI unit 10 of FIG. 1 . In the embodiment, the input image is first interpolated by a first upsampling unit 100 with an interpolation function H, therefore resulting in an interpolated image. For example, if the upsampling factor is 2, a pixel in the input image may correspond to 2×2 pixels in the interpolated image as shown in FIG. 3 , where P denotes the original pixel, and A, B and C denote the interpolated pixels. The interpolated image is then down sampled (or sub-sampled) by a down-sampling unit 102 , therefore resulting in several down-sampled images (e.g., L 1 , L 2 and L 3 ). In the embodiment, the down-sampling of the down-sampling unit 102 is performed by pixel shifting. Regarding the example as shown in FIG. 3 , the down-sampled images L 1 , L 2 and L 3 corresponding to the pixels A, B and C may be expressed follows:

L 1 ( x,y )= H (2 x+ 1,2 y )

L 2 ( x,y )= H (2 x, 2 y+ 1)

L 3 ( x,y )= H (2 x+ 1,2 y+ 1).

Afterwards, the down-sampled images are second interpolated by second upsampling units 104 with the interpolation function H, respectively, therefore resulting in double-interpolated images G (e.g., G 1 , G 2 and G 3 corresponding to L 1 , L 2 and L 3 respectively).

The double-interpolated images G are respectively compared with the input image by difference units 106 , each of which generates an associated double-interpolated difference D (e.g., D 1 , D 2 or D 3 ) that is the pixel value difference between the input image and the corresponding double-interpolated image G. Regarding the example as shown in FIG. 3 , for each pixel in the input image L, the double-interpolated differences D 1 , D 2 and D 3 corresponding to the pixels A, B and C may be expressed follows:

D (2 x, 2 y )=0

D (2 x+ 1,2 y )=| L ( x,y )− G 1 (2 x− 1,2 y )|

D (2 x, 2 y+ 1)=| L ( x,y )− G 2 (2 x, 2 y− 1)|

D (2 x+ 1,2 y+ 1)=| L ( x,y )− G 3 (2 x− 1,2 y− 1)|

Subsequently, a double-interpolated (DI) image selector 108 is configured to select the double-interpolated image G that has the associated minimum double-interpolated difference D. Both the double-interpolated image G and its associated minimum double-interpolated difference D are thus provided, as the outputs of each DI unit 10 ( FIG. 1 ). According to the embodiment, by means of the double-interpolated differences (within a DI unit 10 ), wrongly interpolated area can thus be highlighted and prevented.

Although specific embodiments have been illustrated and described, it will be appreciated by those skilled in the art that various modifications may be made without departing from the scope of the present invention, which is intended to be limited solely by the appended claims.