Image transformation method device for obtaining a three dimensional image

This application claims the priority benefit of Taiwan application serial no. 98120873, filed Jun. 22, 2009. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of specification.

1. Field of the Invention

The present invention relates to an image transformation method and more particularly to a method for obtaining a three dimensional (3D) image using a two dimensional (2D) image and a corresponding depth image.

2. Description of Related Art

In a 3D image display, a barrier and a view design is often used in combination with binocular parallax to enable the human eyes to sense a 3D image. FIG. 1 illustrates a structural view of a 3D image display adopting two view images. Referring to FIG. 1 , a liquid crystal display (LCD) panel 104 has a plurality of pixels V 11 ˜V 15 and V 21 ˜V 24 disposed thereon, wherein the pixels V 11 ˜V 15 comprise a first view image and the pixels V 21 ˜V 24 comprise a second view image. In addition, a backlight module 103 is disposed at a side of the LCD panel 104 and a barrier 105 is disposed at the other side of the LCD panel 104 , wherein the barrier 105 is used to perform a spectrometric process. According to the above, a left eye 101 sees the pixels V 11 ˜V 15 used to form the first view image through the barrier 105 and a right eye 102 sees the pixels V 21 ˜V 24 used to form the second view image through the barrier 105 , which accordingly, allows the human eyes to feel a 3D image by binocular parallax.

In a conventional process of making a 3D image, images or view suitable for the left and the right eyes are read from a memory and the two images are processed and outputted. However, such method requires a significant amount of memory space and consumes more resources. Furthermore, the afore-mentioned images usually undergo a motion process such that the images have holes in part of the pixels. The conventional technology solves the problem of pixel holes by using copy interpolation to interpolate the pixel holes, which, however, generates less smooth and less natural images.

An exemplary embodiment of the present invention provides an image transformation method in which a 3D image is obtained according to a 2D image and a corresponding depth image.

An exemplary embodiment of the present invention provides an image transformation method adapted to an image display device. The image transformation method includes obtaining a 2D image and a corresponding depth image. According to the depth image and N gain values GM W , a motion process is performed on the 2D image to obtain N motion images, wherein a pixel motion value of each of the motion images varies with the corresponding gain value GM W , wherein N and w are positive integers and 1≦w≦N. A plurality of corresponding view images are obtained by respectively performing an interpolation process on the motion images. A synthesis process is performed on the view images to obtain a 3D image.

In one exemplary embodiment of the present invention, the afore-mentioned motion process calculates the pixel motion value in the motion images with the following formula:

S w (i,j)=(D(i,j)/a)*GM w , wherein Sw(i,j) represents the pixel motion value of the w th motion image, (i,j) represents coordinates of a pixel at the i th column and j th row in the w th motion image, D(i,j) represents a pixel value of the depth image, a is a constant, and i and j are positive integers.

In one exemplary embodiment of the present invention, the afore-mentioned constant a is an integer greater than 0.

In one exemplary embodiment of the present invention, the afore-mentioned plurality of motion images are obtained by moving pixel positions in the 2D image according to the corresponding pixel motion value Sw(i,j).

In one exemplary embodiment of the present invention, the afore-mentioned pixel motion value Sw(i,j) represents an amount of motion of the pixel to the left or the right.

In one exemplary embodiment of the present invention, the above-mentioned step of respectively performing the interpolation process on the motion images to obtain a plurality of corresponding view images includes selecting an average of a plurality of pixels adjacent to the pixel hole to interpolate the pixel hole according to a pixel motion direction of each motion image.

In one exemplary embodiment of the present invention, the above-mentioned step of respectively performing the interpolation process on the motion images to obtain a plurality of corresponding view images includes selecting a median of a plurality of pixels adjacent to the pixel hole to interpolate the pixel hole according to a pixel motion direction of each motion image.

In one exemplary embodiment of the present invention, the above-mentioned step of performing a synthesis process on the view images to obtain a 3D image includes displaying the view images on the display device according to the pixel positions corresponding to the view images.

In one exemplary embodiment of the present invention, a computer program product is provided which when loaded in a computer executes the following steps. A 2D image and a corresponding depth image are obtained. According to the depth image and N gain values GM W , a motion process is performed on the 2D image to obtain N motion images, wherein a pixel motion value of each of the motion images varies with the corresponding gain value GM W , wherein N and w are positive integers and 1≦w≦N. A plurality of corresponding view images are obtained by respectively performing an interpolation process on the motion images. A synthesis process is performed on the view images to obtain a 3D image.

According to the above, the present invention provides an image transformation method in which a 3D image is obtained according to a 2D image and a corresponding depth image. In addition to obtaining a plurality of smooth and natural view images which are synthesized as a 3D image, the present invention also conserves memory usage. The image transformation method of the present invention is applicable for image display devices, computer accessible recording media, computer program products, or embedded systems.

In order to make the aforementioned and other features and advantages of the present invention more comprehensible, several embodiments accompanied with figures are described in detail below.

The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

FIG. 1 illustrates a structural view of a 3D image display adopting two view images.

FIG. 2 is a schematic view of an image transformation method according to one embodiment of the present invention.

FIG. 3 is a flow chart of an image transformation method according to one embodiment of the present invention.

FIG. 4A to FIG. 4G are schematic views showing the results of an image transformation method adopting two view images according to one embodiment of the present invention.

FIG. 5A to FIG. 5B are schematic views illustrating an interpolation process of an image transformation method according to one embodiment of the present invention.

FIG. 6A to FIG. 6D are schematic views showing the result of an interpolation process.

FIG. 7A to FIG. 7B are schematic views illustrating a synthesis process of an image transformation method according to one embodiment of the present invention.

FIG. 8A to FIG. 8M are schematic views showing the results of an image transformation method adopting six view images according to another embodiment of the present invention.

FIG. 2 is a schematic view of an image transformation method according to one embodiment of the present invention. Referring to FIG. 2 , a plurality of view images 203 are obtained after a motion process and an interpolation process are performed on a 2D image 201 and a corresponding depth image 202 . Each view image of the view images 203 has different pixel shift as compared to the 2D image 201 . Furthermore, the images of the view images 203 are synthesized to provide a 3D image 204 to a display panel.

FIG. 3 is a flow chart of an image transformation method according to one embodiment of the present invention. Referring to FIG. 3 , a 2D image and a depth image are first received (step S 301 ). By using the depth image and a plurality of gain values, a motion process is performed on the 2D image to obtain a plurality of motion images (step S 302 ). Subsequently, a plurality of corresponding view images are obtained by respectively performing an interpolation process on the motion images (step S 303 ). Finally, a synthesis process is performed on the view images to obtain a 3D image (step S 304 ). Furthermore, the image transformation method of the present embodiment is applicable for an image display device, wherein the image display device displays the 3D image after the synthesis process. The image transformation method of the present embodiment is also applicable for a computer accessible recording medium, wherein the computer accessible recording medium is used to store programs which implement the image transformation method of the present embodiment. On the other hand, the image transformation method of the present embodiment is also further applicable for a computer program product which when loaded in a computer executes the image transformation method of the present embodiment. In addition, the image transformation method of the present embodiment is also applicable for an embedded system, wherein the embedded system includes a processor which executes program codes to implement the image transformation method of the present embodiment and a memory which stores the program codes to implement the image transformation method of the present embodiment. The above descriptions are given for the purpose of exemplifying the devices for implementing the image transformation method of the present embodiment and are not intended to limit the implementation methods thereto.

FIG. 4A to FIG. 4G are schematic views showing the results of an image transformation method adopting two view images according to one embodiment of the present invention. FIG. 4A illustrates a 2D image 401 received in step S 301 and FIG. 4B shows a depth image 402 received in step S 301 , wherein the depth image 402 has depth information of objects in the 2D image 401 . For example, the people objects in the 2D image 401 have lower depth and the background has higher depth. Moreover, the depth image 402 may comprise a plurality of depth values D(i,j), wherein i and j indicate the pixel coordinates in the depth image 402 of the depth value D(i,j), i and j being positive integers. In addition, the depth value D(i,j) is a gray value. In other words, the depth value D(i,j) is a positive integer and 0≦D(i,j)≦255.

In step S 302 , using the depth image and the N gain values GM W , the motion process is performed on the 2D image to obtain the N motion images, wherein the pixel motion value of the N motion images varies with the corresponding gain value GM W , wherein N and w are positive integers and 1≦w≦N. In detail, S w (i,j) represents the pixel motion value of the pixel at the i th column and j th row in the w th motion image. Furthermore, p(i,j) and m w (i,j) respectively represent the pixel values of the pixels at the i th column and j th row of the 2D image 401 and the first motion image 403 , (i,j) are corresponding coordinates, and w=1 and i and j are positive integers. As such, a motion process is performed on the pixel value p(i,j) according to the pixel motion value S w (i,j) to obtain the pixel value m w (i,j), wherein the pixel motion value S w (i,j) varies with the gain value GM w . In other words, the N motion images are obtained by moving pixel positions in the 2D image according to the corresponding pixel motion value S w (i,j). For example, in order to obtain the first motion image 403 , assume the pixel motion value S 1 (11,12)=2 (S 1 (11,12) corresponds to the pixel value p(11,12)) and then the pixel value p(11,12) is shifted to right by two pixels. That is to say, after the motion process, the pixel value m 1 (13,12) is the pixel value p(11,12). Furthermore, if right shift direction is adopted as the pixel motion direction, then m w (i+S w (i,j),j)=p(i,j), i.e. the pixel motion value S w (i,j) represents the pixel motion value to the right.

Furthermore, the pixel motion value S w (i,j) is obtained according to the depth image 402 and the gain value GM w , wherein S w (i,j)=(D(i,j)/a)*GM w , the constant a and the gain value GM w may be obtained based on design experience. For example, the embodiment illustrated in FIG. 4A to FIG. 4G adopts two view images. Therefore, w=1, 2. In addition, assume the constant a=1, GM 1 =2, and GM 2 =4. According to the above, the motion process is performed on the 2D image 401 according to the depth value D(i,j) of the depth image 402 and the gain value GM 1 , i.e. according to the pixel motion value S i (i,j)=(D(i,j)/a)*GM 1 , to obtain the first motion image 403 . In addition, since GM 1 =2, the pixel motion value S 1 (i,j) of the first motion image 403 is a multiple of 2. In other words, each pixel value p(i,j) in the 2D image 401 moves by multiples of 2 to obtain the first motion image 403 and the distance of the motion position is related to the depth value D(i,j).

On the other hand, the motion process is performed on the 2D image 401 according to the depth value D(i,j) and the gain value GM 2 , i.e. according to the pixel motion value S 2 (i,j)=(D(i,j)/a)*GM 2 , to obtain the second motion image 405 in FIG. 4E . In addition, since GM 2 =4, the pixel motion value S 2 (i,j) of the second motion image 405 is a multiple of 4. In other words, each pixel value p(i,j) in the 2D image 401 moves by multiples of 4 to obtain the second motion image 405 and the distance of the motion position is related to the depth value D(i,j). To carry it further, the above-mentioned pixel motion value S w (i,j) may be implemented by way of scale or vector. Directions of the pixel motion value S w (i,j) may be right shift, left shift, up shift, down shift, vector shift, or any combination of the aforesaid pixel motion directions. For example, if a left shift pixel motion direction is adopted, then m w (i−S w (i,j),j)=p(i,j).

The constant a is used to quantify the depth value D(i,j). For example, the constant a may be a positive integer greater than 0. Assume the depth value D(i,j) is a gray value of a positive integer, 0≦D(i,j)≦255, and a=64, then 0≦D(i,j)/a<4. Furthermore, assume D(i,j)/a is truncated, the result of truncating D(i,j)/a is a positive integer greater than or equal to 0 and smaller than or equal to 4. According to the above, the constant a may be used to adjust motion distance of the pixel motion value S w (i,j).

The motion images after the motion process may have pixel holes. For example, the first motion image 403 may have a pixel hole 410 , i.e. the diagonal lines in the first motion image 403 . Similarly, the diagonal lines in the second motion image 405 are also pixel holes. Specifically, in the motion process in step S 302 , not all pixel values of the first motion image 403 are obtained through the pixel motion value S w (i,j) of the 2D image 401 . In other words, pixel values at some positions in the first motion image 403 are not defined and the positions of these undefined pixel values are pixel holes. To interpolate the pixel holes, in step S 303 , a plurality of corresponding view images are obtained by performing an interpolation process on the motion images.

FIG. 5A to FIG. 5B are schematic views illustrating an interpolation process of an image transformation method according to one embodiment of the present invention. Referring to FIG. 5A , a partial block 501 of the motion image includes a plurality of pixel values L 1 ˜L 6 and a pixel hole H. To interpolate the pixel hole H, an interpolation method adopted in the present embodiment may be an average interpolation or a median interpolation. In detail, assume the direction of the above-mentioned motion process adopts right shift. The interpolation may be performed by using a plurality of pixel values (e.g. pixel values L 1 ˜L 6 in FIG. 5A ) located to the left of the pixel hole H and in the proximity of the pixel hole H. In the average interpolation, the pixel value of the pixel hole H is an average of the pixel values L 1 ˜L 6 while in the median interpolation, the pixel value of the pixel hole H is a median of the pixel values L 1 ˜L 6 . Furthermore, the interpolation process may select a plurality of pixel values adjacent to the pixel hole to interpolate the pixel hole according to the pixel motion direction adopted in the motion image. For example, six pixel values are used in FIG. 5A and a partial block 502 of the motion image in FIG. 5B uses 9 pixel values. The motion image after the interpolation is the view image. The number of the pixel values for use may be obtained from design experience. In addition, the interpolation adopted in the present embodiment is not necessarily limited to the average interpolation or the median interpolation.

FIG. 6A to FIG. 6D are schematic views illustrating the result of the interpolation process, wherein the motion image after the interpolation process is the view image. A motion image 601 in FIG. 6A includes a pixel hole 611 . A view image 602 in FIG. 6B is obtained by performing conventional copy interpolation on the motion image 601 , a view image 603 in FIG. 6C is obtained by performing the median interpolation on six pixel values of the motion image 601 , and a view image 604 in FIG. 6D is obtained by performing the average interpolation on six pixel values of the motion image 601 , wherein a position of a block 612 in the view image 602 , a position of a block 613 in the view image 603 , and a position of a block 614 in the view image 604 correspond to the position of the pixel hole 611 in the motion image 601 . It can be seen from FIG. 6A to FIG. 6D that the view image 603 obtained by the median interpolation and the view image 604 obtained by the average interpolation appear to be smoother and more natural than the view image 602 obtained by the conventional copy interpolation.

According to the above interpolation, the first view image 404 in FIG. 4D is obtained by performing the average interpolation on six pixel values of the first motion image 403 in FIG. 4C while the second view image 406 in FIG. 4F is obtained by performing the average interpolation on six pixel values of the second motion image 405 in FIG. 4E .

In step S 304 , the synthesis process is performed on the view images to obtain the 3D image, wherein the synthesis process is designed in accordance with the panel. FIG. 7A to FIG. 7B are schematic views illustrating a synthesis process of an image transformation method according to one embodiment of the present invention. FIG. 7A illustrates a partial block 701 of the panel (not shown), wherein the panel adopts two view images, red columns R 1 and R 2 , green columns G 1 and G 2 , and blue columns B 1 and B 2 represent individual columns of the partial block 701 , synthesis points V 1 represent pixel values to be filled at corresponding pixel positions in the first view image 404 , and synthesis points V 2 represent pixel values to be filled at corresponding pixel positions in the second view image 406 . Specifically, the red column R 1 sequentially fills pixel values at the corresponding positions in the first view image 404 , the green column G 1 sequentially fills pixel values at the corresponding positions in the second view image 406 , the blue column B 1 sequentially fills pixel values at the corresponding positions in the first view image 404 , and so on. Furthermore, the above-mentioned synthesis process is performed on the motion images 403 and 405 to obtain a 3D image 407 in FIG. 4G .

Based on the above image transformation method adopting two view images, embodiments of the present invention may also adopt six view images. FIG. 8A to FIG. 8M are schematic views showing the results of an image transformation method adopting six view images according to another embodiment of the present invention. According to the image transformation method of the present embodiment, in step S 302 , the depth image 402 and the gain values GM w are used to perform the motion process on the 2D image 401 and to obtain the motion images, wherein w in the gain value GM w may be assumed to be 1˜6. Accordingly, the motion process is performed on the 2D image 401 according to the depth value D(i,j) of the depth image 402 and the gain value GM 1 , i.e. according to the pixel motion value S 1 (i,j)=(D(i,j)/a)*GM 1 to obtain the motion image in FIG. 8A , according to the pixel motion value S 2 (i,j)=(D(i,j)/a)*GM 2 to obtain the motion image in FIG. 8C , according to the pixel motion value S 3 (i,j)=(D(i,j)/a)*GM 3 to obtain the motion image in FIG. 8E , according to the pixel motion value S 4 (i,j)=(D(i,j)/a)*GM 4 to obtain the motion image in FIG. 8G , according to the pixel motion value S 5 (i,j)=(D(i,j)/a)*GM 5 to obtain the motion image in FIG. 8I , and according to the pixel motion value S 6 (i,j)=(D(i,j)/a)*GM 6 to obtain the motion image in FIG. 8K .

On the other hand, according to the image transformation method of the present embodiment, subsequently in step S 303 , the interpolation process is performed on the above motion images to obtain the plurality view images. For example, the interpolation process may adopt the average interpolation on six pixel values. As such, the average interpolation of six pixel values is performed on the motion image of FIG. 8A to obtain the view image of FIG. 8B and the average interpolation is performed on the motion image of FIG. 8C to obtain the view image of FIG. 8D . Similarly, the view image of FIG. 8F corresponds to the motion image of FIG. 8E , the view image of FIG. 8H corresponds to the motion image of FIG. 8G , the view image of FIG. 8J corresponds to the motion image of FIG. 8I and the view image of FIG. 8L corresponds to the motion image of FIG. 8K .

According to the image transformation method of the present embodiment, subsequently in step S 304 , the synthesis process is performed on the view images to obtain the 3D image. In other words, the synthesis process is performed on the view images shown in FIGS. 8B , 8 D, 8 F, 8 H, 8 J, and 8 L to obtain the 3D image shown in FIG. 8M . In detail, FIG. 7B illustrates a partial block 702 of the panel (not shown), wherein the panel adopts six view images. Red columns R 1 and R 2 , green columns G 1 and G 2 , and blue columns B 1 and B 2 represent individual columns of the partial block 702 . Accordingly, in the synthesis process, synthesis points V 1 represent pixel values to be filled at corresponding pixel positions in the view image of FIG. 8B , synthesis points V 2 represent pixel values to be filled at corresponding pixel positions in the view image of FIG. 8D , synthesis points V 3 represent pixel values to be filled at corresponding pixel positions in the view image of FIG. 8F , synthesis points V 4 represent pixel values to be filled at corresponding pixel positions in the view image of FIG. 8H , synthesis points V 5 represent pixel values to be filled at corresponding pixel positions in the view image of FIG. 8J , synthesis points V 6 represent pixel values to be filled at corresponding pixel positions in the view image of FIG. 8L . According to the above synthesis process, the 3D image of FIG. 8M is obtained.

According to the above, the view images obtained in the embodiments of the present invention are smoother and more natural than the view images obtained by conventional interpolation. Moreover, the embodiments of the present invention also have the advantage of conserving memory usage. In a conventional process of making a 3D image, images or views suitable for the left and the right eyes are read from a memory and the two images are processed and outputted. According to descriptions in the above embodiments, a 2D image and a depth image are received from a memory and a motion process is used to obtain a plurality of motion images for subsequent processes. As such, when the panel adopts more view images, e.g. six view images, then six images suitable for the left eye and six images suitable for the right eye need to be read in the conventional method for making a 3D image for the subsequent synthesis process. Compared to the conventional technology, in the above embodiments, only a 2D image and a depth image need to be received from the memory and a motion process is used to obtain six motion images for subsequent processes. Then, interpolation is performed on the six motion images to obtain six view images, wherein when each view image is generated, data of each view image covers the corresponding motion image, thereby reducing memory usage. Moreover, in the above embodiments, the required memory space does not increase because the panel uses more view images.

According to the above, the present invention provides an image transformation method in which a 3D image is obtained according to a 2D image and a corresponding depth image. First, by using the depth image and a plurality of gain values, a motion process is performed on the 2D image to obtain a plurality of motion images. Subsequently, an interpolation process is performed on the motion images to thereby obtain a plurality of smooth and natural view images in addition to reducing memory usage. Finally, a synthesis process is performed on the view images to obtain a 3D image. The image transformation method of the present invention is applicable for image display devices, computer accessible recording media, computer program products, or embedded systems.

Although the present invention has been described with reference to the above embodiments, it will be apparent to one of the ordinary skill in the art that modifications to the described embodiment may be made without departing from the spirit of the invention. Accordingly, the scope of the invention will be defined by the attached claims not by the above detailed descriptions.