Method for detecting and removing scrolling texts during video communication

The present invention relates to a video screen optimization method, and more particularly to a method for detecting and removing scrolling texts during video communication.

With the rapid development of video and network technologies, there are a lot of applications related to video communication in our lives recently, such as video conferencing, internet video courses, internet TV and digital TV. Video images usually include a main screen, background and texts to provide information for audiences. In order to emphasize certain information or provide certain advertisements, scrolling texts will be inserted at the periphery of the video screen, so that the audience can see the information in addition to the video programs. However, many audiences would be distracted because of the scrolling texts and cannot focus on the program. If the audience can choose whether he/she wants to see the scrolling texts and effectively hide the scrolling texts, the viewing effects can be significantly enhanced. This is actually the technical problem the present invention wants to solve.

The technical problem the present invention wants to solve is described above and the present invention provides a method for detecting and removing scrolling texts during video communication.

The present invention provides an adaptive transient difference processing method of the video communication, especially the calculation of the video frame. The video communication signal is converted from color to grayscale signal, and the noise of the video is reduced through the low-pass filter to smooth the video images and further the accuracy of the adaptive transient difference processing. Using the adaptive transient difference processing ( 10 ) to take first N frames f jk t−N and a current frame f jk t , and subtract them to obtain a frame difference Diff jk =|f jk t −f jk t−N |. If the frame difference Diff jk is greater than a threshold value, it is determined that the current frame f jk t has scrolling texts. On the contrary, if no scrolling texts found and the threshold value to detect scrolling texts is the localized difference average of the size of an L×K window, the localized difference can be obtained as

LM ij = ∑ i = - K K ⁢ ∑ j = - L L ⁢ Diff i + K , j + L K × L

to compare with the threshold value. The greater the localized difference LM ij , the smaller the threshold value of scrolling text detection. When the frame difference Diff jk of a frame of one unit point is greater than the threshold value, the unit point is a scrolling text point. In one embodiment, the present invention uses the adaptive transient difference processing to detect scrolling texts during video communication and interpolating the first N frames before the current position of the scrolling texts to replace the current frame f jk t to achieve the goal of hiding the scrolling texts during video communication to enhance the viewing effect.

In another embodiment, the adaptive transient difference processing uses adaptive transient difference calculation method, wherein if the first N frame f jk t 's average frame difference ADiff jk is smaller than the threshold value, a further calculation of the first N+1 frames is conducted until the average frame difference ADiff jk is greater than the threshold value. Taking the position of the first N frames f jk t−N that is greater than the threshold value as the next calculation value of the difference between the current frame f jk t and the frame difference Diff jk .

The main object of the present invention is that the calculation takes the first N frames f jk t−N and compares with the current frame f jk t to obtain the frame difference Diff jk , and if the frame difference Diff jk is greater than the threshold value, it is determined that the current frame f jk t has scrolling texts. The current frame f jk t can be replaced by interpolation with the first N frames f jk t−N at the same position of the scrolling texts to hide the scrolling texts and enhance the viewing effect.

The secondary object of the present invention is that the frame difference Diff jk calculated by the adaptive transient difference processing is 30% of the periphery area of video communication screen. The average frame difference ADiff jk is obtained by accumulating the frame difference Diff jk , and if the average frame difference ADiff jk is greater than the threshold value, it is determined that the periphery area of video communication screen has scrolling texts, so as to increase the accuracy of detecting the scrolling texts and decrease the calculation difficulty.

Another object of the present invention is that when hiding the scrolling texts, the calculation can be expanded N×N around the periphery of the texts, so that the cross edge of every point of the scrolling is considered an area to be hidden to select the peripheral area of the entire scrolling texts to completely hide the scrolling texts, and further eliminate the background that resembles the scrolling texts to enhance the hiding effect.

FIG. 1 illustrates a flow diagram of the method for detecting and removing scrolling texts in the present invention.

FIG. 2 illustrates a schematic view of detecting the periphery of video communication in the present invention.

FIG. 3 illustrates a flow diagram of the adaptive transient difference processing method in the present invention.

FIG. 4 illustrates a schematic view of expanding area of scrolling texts in the present invention.

FIG. 5 illustrates a flow diagram of removing scrolling texts in the present invention.

The detailed description set forth below is intended as a description of the presently exemplary device provided in accordance with aspects of the present invention and is not intended to represent the only forms in which the present invention may be prepared or utilized. It is to be understood, rather, that the same or equivalent functions and components may be accomplished by different embodiments that are also intended to be encompassed within the spirit and scope of the invention.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood to one of ordinary skill in the art to which this invention belongs. Although any methods, devices and materials similar or equivalent to those described can be used in the practice or testing of the invention, the exemplary methods, devices and materials are now described.

All publications mentioned are incorporated by reference for the purpose of describing and disclosing, for example, the designs and methodologies that are described in the publications that might be used in connection with the presently described invention. The publications listed or discussed above, below and throughout the text are provided solely for their disclosure prior to the filing date of the present application. Nothing herein is to be construed as an admission that the inventors are not entitled to antedate such disclosure by virtue of prior invention.

In order to further understand the goal, characteristics and effect of the present invention, a number of embodiments along with the drawings are illustrated as following:

Referring to FIG. 1 , a method for detecting and removing scrolling text during video communication is disclosed. The present invention is based on an adaptive transient difference processing ( 10 ) of the video communication, especially the calculation of the video frame. The video communication signal is converted from color to grayscale signal, and the noise of the video is reduced through the low-pass filter to smooth the video images and further the accuracy of the adaptive transient difference processing ( 10 ). Using the adaptive transient difference processing ( 10 ) to take first N frames f jk t−N and a current frame f jk t , and subtract them to obtain a frame difference Diff jk =|f jk t −f jk t−N |. If the frame difference Diff jk is greater than a threshold value, it is determined that the current frame f jk t has scrolling texts. On the contrary, if no scrolling texts found and the threshold value to detect scrolling texts is the localized difference average of the size of an L×K window, the localized difference can be obtained as

LM ij = ∑ i = - K K ⁢ ∑ j = - L L ⁢ Diff i + K , j + L K × L

to compare with the threshold value. The greater the localized difference LM ij , the smaller the threshold value of scrolling text detection. When the frame difference Diff jk of a frame of one unit point is greater than the threshold value, the unit point is a scrolling text point. The threshold can also be a fixed point which can be manually set. In summary, using the adaptive transient difference processing ( 10 ) to detect scrolling texts during video communication and interpolating the first N frames before the current position of the scrolling texts to replace the current frame f jk t to achieve the goal of hiding the scrolling texts during video communication to enhance the viewing effect.

Referring again to FIGS. 1 and 2 , the method for detecting scrolling text during video communication is further illustrated. Since the scrolling texts appear more on the periphery of the video communication screen, the frame difference Diff jk of using the adaptive transient difference processing ( 10 ) is to detect 30% of the area of the periphery of the video communication screen ( 20 ) and accumulate the average frame difference Diff jk to obtain an average frame difference

ADiff jk = ∑ j = 0 N ⁢ ∑ k = 0 M ⁢ Diff jk N × M .

If the average frame difference ADiff jk is greater than the threshold value, it is determined that the periphery area of video communication screen ( 20 ) has scrolling texts, so as to enhance the accuracy of detecting the scrolling texts and decrease the calculation difficulty. Furthermore, the adaptive transient difference processing ( 10 ) uses adaptive transient difference calculation as shown in FIG. 3 , wherein the calculation uses the initial frame (N=1) of video communication as the first N frames f jk t−N , and obtains the average frame difference ADiff jk using the first N frames f jk t−N and the current frame f jk t , then checking if the average frame difference ADiff jk is greater than the threshold value. When the average frame difference ADiff jk obtained from the first N frames f jk t−N is smaller than the threshold value, a further calculation of the first N+1 frames is conducted until the average frame difference ADiff jk is greater than the threshold value. Taking the position of the first N frames f jk t−N that is greater than the threshold value as the next calculation value of the difference between the current frame f jk t and the frame difference Diff jk , and when N of the N th frame f jk t−N is the maximum, it is determined that there is no scrolling text.

Referring again to FIG. 1 , the method for detecting hidden scrolling text during video communication is further illustrated. The video communication can be converted from color signals to grayscale signals to reduce calculation complexity of the adaptive transient difference processing and increase the accuracy of determining scrolling texts. Referring to FIG. 4 , when the scrolling texts are hidden, an N×N expanded calculation is conducted at the periphery of the texts, so that the cross edge of every point of the scrolling is considered an area to be hidden to select the peripheral area of the entire scrolling texts to completely hide the scrolling texts, and further eliminate the background that resembles the scrolling texts. Referring to FIG. 5 , the method of hiding scrolling text during video communication can be conducted in either color or grayscale mode. The grayscale mode is to calculate the grayscale value of the relative position point of the first N frames f jk t−N and the grayscale value of the scrolling texts of the current frame f jk t , and if the difference of the two grayscale values is greater than the threshold value, the first N frames f jk t−N is interpolated as background to hide the scrolling texts. The color mode is to calculate the RGB value of the relative position point of the first N frames f jk t−N and the RGB value of the scrolling texts of the current frame f jk t , and if the difference of the two RGB values is greater than the threshold value, the first N frames f jk t−N is interpolated as background to hide the scrolling texts. If the calculation of the RGB value of the relative position point of the first N frames f jk t−N is not yet greater than the threshold value, using the current frame f jk t to conduct bilinear interpolation on the background of the periphery of the scrolling texts to hide the scrolling texts.

According to the embodiments described above, the present invention has the following advantages: (1) the calculation takes the first N frames f jk t−N and compares with the current frame f jk t to obtain the frame difference Diff jk , and if the frame difference Diff jk is greater than the threshold value, it is determined that the current frame f jk t has scrolling texts. The current frame f jk t can be replaced by interpolation with the first N frames f jk t−N at the same position of the scrolling texts to hide the scrolling texts and enhance the viewing effect; (2) the frame difference Diff jk calculated by the adaptive transient difference processing ( 10 ) is 30% of the periphery area of video communication screen ( 20 ). The average frame difference ADiff jk is obtained by accumulating the frame difference Diff jk , and if the average frame difference ADiff jk is greater than the threshold value, it is determined that the periphery area of video communication screen ( 20 ) has scrolling texts, so as to increase the accuracy of detecting the scrolling texts and decrease the calculation difficulty; and (3) when hiding the scrolling texts, the calculation can be expanded N×N around the periphery of the texts, so that the cross edge of every point of the scrolling is considered an area to be hidden to select the peripheral area of the entire scrolling texts to completely hide the scrolling texts, and further eliminate the background that resembles the scrolling texts to enhance the hiding effect.

Having described the invention by the description and illustrations above, it should be understood that these are exemplary of the invention and are not to be considered as limiting. Accordingly, the invention is not to be considered as limited by the foregoing description, but includes any equivalents.