Deinterlacing apparatus of digital image data

1. Field of the Invention

The present invention relates to digital images and more particularly to a deinterlacing apparatus of digital image data by obtaining motion information and executing interpolations according to the motion information.

2. Discussion of Related Art

Generally, a picture of a digital TV may be displayed on a monitor by interlaced scanning or progressive scanning. Some display monitors require one type of scanning pattern while the image is received in the other type of scanning pattern. Therefore, a deinterlacing apparatus converts an interlaced field to a non-interlaced field (or progressive frame) data by filling pixel values of omitted field lines in the interlaced field.

Particularly, a digital TV supports a display of progressive frame data. As a result, if an interlaced field is received, the data must be converted into the progressive frame data. Typical methods used for converting the interlaced field into the progressive frame are an inter-field interpolation method which inserts data from a previous field between current field line data, without motion compensation, and a line doubling method which repeatedly uses field line data of a current field. There are other methods such as an intra-field interpolation method which uses a linear line interpolation of the current field.

FIG. 1A shows the line doubling method, FIG. 1B shows the inter-field interpolation method, and FIG. 1C shows the intra-field interpolation method. Such interpolation methods can be constructed by simple hardware, but may cause degradation to the image. Namely, in the line doubling method, the image quality of displayed video would generally be degraded after an interpolation. In the inter-field interpolation method without the motion compensation, the image quality of video portions with motion may considerably be degraded after an interpolation. In contrast, the image quality of video portions without motion may be degraded after interpolation by the intra-field interpolation method.

Accordingly, a method of detecting a motion and converting the interlaced field to a progressive frame has been proposed by Bernard in U.S. Pat. No. 5,027,201 and by Faroudja in U.S. Pat. No. 5,159,451. Although these methods improve the video quality after interpolation, a complicated hardware including a plurality of memories and a complicated processing are required. As a result, the overall cost of manufacturing a deinterlacing apparatus would increase, thereby increasing the cost of a digital TV.

Accordingly, an object of the present invention is to solve at least the problems and disadvantages of the related art.

An object of the present invention is to provide an efficient deinterlacing method and apparatus.

Another object of the present intention is to provide a deinterlacing method and apparatus for digital image data, which is capable of improving a quality of the displayed video.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objects and advantages of the invention may be realized and attained as particularly pointed out in the appended claims.

To achieve the objects and in accordance with the purposes of the invention, as embodied and broadly described herein, a deinterlacing apparatus includes a BD & BPPD detecting unit for calculating a brightness profile pattern difference and a brightness difference between a current field and a previous field data; a BD & BPPD combiner for respectively limiting the obtained BD and the BPPD by a threshold value, mapping the BD and BPPD to preset levels, and deciding an extent of motion present according to its magnitude; a median filter for eliminating a noise component of the extent of motion value output by the BD & BPPD combiner, and grouping portions with motion; a motion extender for extending an effect of a motion to adjacent pixels; and an edge detector for detecting an edge direction for a pixel to be interpolated in a current field, detecting a characteristic of a local correlation in the vertical direction, and providing additional information necessary for a detection of intra-field motion information and a calculation of intra-field motion extent.

The invention will be described in detail with reference to the following drawings in which like reference numerals refer to like elements wherein:

FIGS. 1A to 1 C show methods of interpolating digital image data in the related art;

FIG. 2 is a block diagram of a deinterlacing apparatus in accordance with the present invention;

FIG. 3 shows a method of detecting a locally vertical correlation in accordance with the present invention;

FIG. 4 shows a constructive diagram for three consecutive field data used in the deinterlacing apparatus of FIG. 2; and

FIGS. 5 and 6 show examples of input interlaced fields.

Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

Generally, a deinterlacing apparatus in which pixels are interpolated according to a detected extent of motion is disclosed in copending application Ser. No. 09/224,049 entitled “Deinterlacing Device and Method for Digital TV receiver,” and is fully incorporated herein. In the deinterlacing method and apparatus according to the present invention, motion information is detected and an appropriate interpolation is performed according to both an extent of motion and a direction of the motion.

Particularly, the present deinterlacing apparatus includes an edge detector which detects a edge direction of an object around pixels to be interpolated and calculates an interpolation value according to the detected direction. The edge direction of an object is a direction toward which an object may be present within a data block of a field line to be interpolated. For example, if a diagonal bar is present from the bottom left corner to the top right corner, the edge direction would be in the 45 degree line.

The edge detector may also extract a local correlation around the pixel to be interpolated. For example, if an object moves rapidly as shown in FIG. 3, a BD and BPPD values between two interlaced fields n and (n−2) around coordinates (i,j) would be ‘0.’ Thus, the video portion for n to (n−2) would be regarded as a portion without motion. To prevent an incorrect detection of motion, a local correlation is obtained to more accurately interpolate the pixels.

Therefore, the edge detector generates information to calculate BD and BPPD values necessary to calculate an intra-field motion information. FIG. 2 is a block diagram of a deinterlacing apparatus according to the present invention including a field data storage unit 21 which stores three fields, namely a current interlaced field n and two previous interlaced fields (n−1) and (n−2); a BD & BPPD detecting unit 22 which calculates a brightness profile pattern difference (BPPD) value and a brightness difference (BD) value between the current interlaced field and the previous interlaced field (n−2) in order to determine a presence and an extent of motion; a BD & BPPD combiner 23 which respectively limits the BD and BPPD values by a threshold value, maps the limited BD and BPPD values to preset levels, compares the magnitudes and decides the presence and extent of the motion; a median filter 24 which eliminates a noise component in the detected motion, and groups portions having the motion; a motion extender 25 which extends an effect of the motion to adjacent pixels; an edge detector 26 which detects an edge direction around a pixel to be interpolated; a soft switch 27 which mixes an output value of the edge detector 26 and a value (c) of the previous interlaced field (n−1) from the same position as that of a pixel to be interpolated, and outputs the mixed value to appropriately interpolate pixel values; and a vertical line converter 28 which converts the number of vertical lines to conform with a required display format, by using data of the current field and interpolation line data from the soft switch 27 , and outputs a non-interlaced field.

The operation of the present deinterlacing apparatus will be explained below with reference to FIG. 3 which show a method of detecting a local correlation and FIG. 4 which sets a diagram for three consecutive field data, where n is a current interlaced field, and (n−1) and (n−2) are previous interlaced fields. First, the edge direction detection to determine an appropriate interpolation of a pixel will be explained with reference to Table 1 below.

To determine the edge direction of a pixel A to be interpolated, the following calculation is executed where |*| represents an absolute value operation and Med(a, b, c) represents a median of a, b, c.

Med −45 ={|P ( i, j− 2)− P ( i+ 1 , j )|, | P ( i, j− 1)− P ( i+ 1 , j+ 1)|, | P ( i, j )− P ( i+ 1 , j+ 2)|}  [1]

Med 45 ={|P ( i, j )− P ( i+ 1 , j− 2)|, | P ( i, j+ 1)− P ( i− 1 , j− 1), | P ( i, j+ 2)− P ( i+ 1 , j )|}  [2]

Med 0 ={|P ( i, j− 1)− P ( i+ 1 , j− 1)|, | P ( i, j )− P ( i+ 1 , j )|, | P ( i, j+ 1)− P ( i+ 1 , j+ 1)|}  [3]

Equation [1] represents an edge in the −45 degree line direction, Equation [2] represents an edge in the 45 degree line direction, and Equation [3] represents an edge in the 0 degree line direction. For each direction, correlations of three pixels are obtained and a median of the three correlations is used to reduce an effect of noise contained in the input field image data.

The values obtained through Equations [1]˜[3] are compared with a threshold value. At this time, mutually different threshold values may be applied according to the respective directions. However, a same threshold E th may be used in all directions, as shown below.

If Med 45 <E th , edge45°=0 else edge45°=Med 45   [4]

If Med 0 <E th , edge0°=0 else edge0°=Med 0   [5]

If Med −45 <E th , edge−45°=0 else edge−45°=Med −45   [6]

A final determination of a edge direction may be represented as follows, where Min{a,b,c} means the minimum value from a,b,c.

Min{edge45°, edge0°, edge−45°}  [7]

When values obtained in all directions are equivalent, the edge exists in the 0° direction and this may be represented as edge(0°). Similarly, if values obtained for 0 degree and 45 degree directions are equivalent, the edges exist in the directions of 45 degree and 0 degree, which are represented as edge(45° and 0°). The possible cases are shown in Table 2, where α<β and l*m*n.

The pixel A would be interpolated as shown in Table 3, according to the detected edge direction in the edge detector for the intra-field interpolation.

It should be noted that the pixel interpolation was decided in consideration of five pixels per line, but can be extended to N number of pixels. Also, only three edge directions were detected in the above embodiment, but more or less edge directions may be detected depending upon the required accuracy. By maintaining a characteristic of an image near an edge through the interpolation pixel value decision as described above, an overall video quality after an interpolation can be significantly improved.

Thereafter, a local correlation may be detected. As shown in FIG. 3, when an object moves rapidly, the BD and BPPD values between a current interlaced field n and a previous interlaced field (n−2) for pixels adjacent the coordinate (i,j) would be ‘0’. Thus, the pixels near coordinate (i,j) of the current field n to be interpolated would be interpolated by pixel values of the previous field (n−1). However, a moving object exists on the coordinate (i,j) of the field n as shown in FIG. 3, which would cause a mistake in the interpolation.

Accordingly, a local correlation in the vertical direction can further be detected to catch rapidly moving objects, thereby allowing a more accurate interpolation. Generally, an object moves in a shape of a lump. In the detection of the local correlation, a line i immediately above and a line i+1 immediately below a line to be interpolated are used. The local correlation is detected using pixels observed around the pixel A to be interpolated. Particularly, three methods can be considered in obtaining the local correlation in the vertical direction V cor depending upon the accuracy required.

In the first method,

if

Med{|P(i, j−1)−P(i+1, j−1)|, |P(i, j)−P(i+1, j)|, |P(i, j+1)−P(i+1, j+1)|}<V th ,

V cor =1

else

V cor =0.

In a second method,

if

|Med{P(i, j−1), P(i, j), P(i, j+1)}−Med{P(i−1, j−1), P(i+1, j), P(i+1, j+1)}|<V th ,

V cor =1

else

V cor =0.

In a third method,

i) if

|Med{P(i, j−2), P(i, j−1), P(i, j)}−Med{P(i, j), P(i, j+1), P(i, j+2)}|<H th ,

Hedge 1 =0

else

Hedge 1 =1;

ii) if

|Med{P(i+1, j−2), P(i+1, j−1), P(i+1, j)}−Med{P(i+1, j), P(i+1, j+1), P(i+1, j+2)}|<H th ,

Hedge 2 =0

else

Hedge 2 =1;

iii) if

|Med{P(i, j−1), P(i, j), P(i, j+1)}−Med{P(i+1, j−1), P(i+1, j), P(i+1, j+1)}|<V th ,

Vedge=0

else

Vedge=1.

In the third method, the local correlation in the vertical direction V cor is decided according to Table 4, after executing i), ii) and iii).

The result entered in Table 4 may also be represented by the following Boolean algebra equation.

Vcor=NOT(Hedge 1 XOR Hedge 2 OR Vedge)  [8]

In the above three methods, V th indicates a vertical direction threshold and H th represents a horizontal direction threshold. Also, a V cor value of ‘0’ indicates that there is no local correlation in the vertical direction, and thus the extent of motion is measured without utilizing a correlation as in the co-pending application Ser. No. 09/224,049. In contrast, a V cor value of ‘1’ means that there is a local correlation in the vertical direction, and the motion extent is measured according to the following process with reference to FIG. 4 .

First, the values BPPD a jk , BPPD b jk ; BD a jk , BD b jk are obtained also as disclosed in co-pending application Ser. No. 09/244,049.

Second, the values BPPD a jk , BPPD b jk ; BD a jk , BD b jk are obtained as follows:

BPPD a jk =|BPP c1 j −BPP a1 k |+|BPP cr j −BPP ar k |;  [9]

BPPD b jk =|BPP c1 j −BPP b1 k |+|BPP cr j −BPP br k |;  [10]

BD a jk =|B c j −B a k |;  [11]

BD b jk =|B c j −B c k |;  [12]

where,

BPP c1 j =B c j −B c1 j ;  [13]

BPP cr j =B c j −B cr j .  [14]

To prevent a degradation in the resolution of an overall video due to a misjudgement of the motion presence, the local correlation in the vertical direction is used as additional information to determine a presence and extent of motion.

This may be performed through the following equation.

BPPD min jk =min{ BPPD a jk ·BPPD b jk }  [15]

BD min jk =min{ BD a jk ·BD b jk }  [16]

BPPD =max{ BPPD a jk ·BPPD b jk ·BPPD min jk }  [17]

BD =max{ BD a jk ·BD b jk ·BD min jk }  [18]

Namely, a presence and extent of motion is re-determined using the local correlation information. Such operation prevents problems caused by incorrect interpolation due to an existence of an object moving on the neighborhood of the coordinates of nth field shown in FIG. 3 .

For example, FIGS. 5 and 6 each show a portion of input data for three interlaced fields. If the pixels of previous fields (n−1) and (n−2) around the pixel position to be interpolated in field n appears as in FIG. 5, the local correlation in the vertical direction V cor =1, but Equations [17] and [18] results in values of ‘0’, indicating that there is actually no motion. In contrast, if the pixels of previous fields (n−1) and (n−2) around the pixel position to be interpolated in field n appears as in FIG. 6, V cor =1 and Equations [17] and [18] would result in a non-zero value, indicating that there is motion. Therefore, a band limitation is appropriately executed so as not to cause a serious intra-field line flicker phenomenon when an original image enters the interlace field.

Namely, if pixels in lines i and (i+1) of the current interlaced nth field have a local correlation for a still image, i.e. portion without motion, pixels in line i of a previous (n−1)th field would also have the local correlation detected for pixels in lines i and i+1 of the nth field. Such operation is executed and judged through Equations [15]˜[18].

In the preferred embodiment, both the edge direction and local correlations are used in converting an interlaced field into a non-interlaced field for a progressive scanning. Moreover, in obtaining the local correlation, the second method requires more operations than the first method, and the third method require more operations than the second method. Namely, more operations means more calculation, but also more accuracy. Therefore, the method for obtaining the local correlation may be selected based upon the needs of the system.

In accordance with the present invention, a deinterlacing apparatus of digital image data has the following effects. An overall screen quality after an interpolation can be improved by maintaining an edge characteristic of an interpolation pixel. In addition, an interlaced field is converted into a progressive field by considering a vertical correlation of observed objects, thereby further improving an overall screen quality. Finally, the structure of the format conversion circuit is simplified to reduce an overall manufacturing expense.

The foregoing embodiments are merely exemplary and are not to be construed as limiting the present invention. The present teachings can be readily applied to other types of apparatuses. The description of the present invention is intended to be illustrative, and not to limit the scope of the claims. Many alternatives, modifications, and variations will be apparent to those skilled in the art.