Audio signal processing apparatus and audio signal processing method

The present invention relates to an audio signal processing apparatus and an audio signal processing method.

In content broadcast on television, human voices such as lines or narration often have a high correlation between left and right channels of a stereo signal. In contrast, background sounds such as BGM often have a low correlation between left and right channels of a stereo signal.

Based on the above premise, there is a technique for improving the ease of hearing human voices by extracting and enhancing the correlation components of the left and right channels of a stereo signal.

For example, Patent Reference 1 discloses a method for enhancing only human voices by applying, to a sum signal of left and right channels of a stereo signal, a filter for extracting a vocal voice band and a notch filter for damping a predetermined frequency component from the vocal voice band.

Patent Reference

Patent Reference 1: Japanese Patent Application Publication No. 2005-086462

Problem to be Solved by the Invention

However, in the prior art, since the correlation component is extracted by using the sum signal of a stereo signal, when there is a deviation of several milliseconds (ms) between the left and right channels of the stereo signal, for example, it is not possible to improve the ease of hearing human voices or the like.

It is therefore an object of one or more aspects of the present invention to improve the ease of hearing human voices even when there is a time axis deviation between the first signal and the second signal.

Means of Solving the Problem

One aspect of the present invention provides an audio signal processing apparatus receiving inputs of a first signal and a second signal, comprising: a first correlation component separating unit configured to predict the first signal from the second signal in a predetermined period to generate a first correlation component signal having a correlation with the first signal in the second signal, and to add a signal having an inverted phase of the first correlation component signal to the first signal to separate, from the first signal, a first non-correlation component signal having no correlation with the second signal; a second correlation component separating unit configured to predict the second signal from the first signal in the predetermined period to generate a second correlation component signal having a correlation with the second signal in the first signal, and to add a signal having an inverted phase of the second correlation component signal to the second signal to separate, from the second signal, a second non-correlation component signal having no correlation with the first signal; a correlation component synthesizing unit configured to synthesize the first correlation component signal and the second correlation component signal to generate a synthesized correlation component signal; a first gain multiplying unit configured to multiply the synthesized correlation component signal by a gain to generate a correlation component signal; a first signal adding unit configured to add the correlation component signal and the first non-correlation component signal; and a second signal adding unit configured to add the correlation component signal and the second non-correlation component signal.

Another aspect of the present invention provides an audio signal processing method comprises: receiving inputs of a first signal and a second signal, predicting the first signal from the second signal in a predetermined period to generate a first correlation component signal having a correlation with the first signal in the second signal; adding a signal having an inverted phase of the first correlation component signal to the first signal to separate, from the first signal, a first non-correlation component signal having no correlation with the second signal; predicting the second signal from the first signal in the predetermined period to generate a second correlation component signal having a correlation with the second signal in the first signal; adding a signal having an inverted phase of the second correlation component signal to the second signal to separate, from the second signal, a second non-correlation component signal having no correlation with the first signal; synthesizing the first correlation component signal and the second correlation component signal to generate a synthesized correlation component signal; multiplying the synthesized correlation component signal by a gain to generate a correlation component signal; adding the correlation component signal and the first non-correlation component signal; and adding the correlation component signal and the second non-correlation component signal.

Effects of the Invention

According to one or more aspects of the present invention, it is possible to improve the ease of hearing human voices even when there is a time axis deviation between the first signal and the second signal.

FIG. 1 is a block diagram schematically illustrating a configuration of an audio signal processing apparatus according to Embodiment 1.

FIG. 2 is a block diagram schematically illustrating a configuration of a first correlation component separating unit.

FIG. 3 is a block diagram schematically illustrating a configuration of a second correlation component separating unit.

FIGS. 4A and 4B are block diagrams illustrating examples of hardware and software configurations of an audio signal processing apparatus.

FIG. 5 is a flowchart indicating a process in an audio signal processing apparatus.

FIG. 6 is a block diagram schematically illustrating a configuration of an audio signal processing apparatus according to Embodiment 2.

FIG. 7 is a schematic diagram illustrating an example of frequency characteristics of a digital filter used for band enhancement.

FIG. 8 is a block diagram schematically illustrating a configuration of an audio signal processing apparatus according to Embodiment 3.

FIG. 1 is a block diagram schematically illustrating a configuration of an audio signal processing apparatus 100 according to Embodiment 1.

The audio signal processing apparatus 100 includes a first correlation component separating unit 110 , a second correlation component separating unit 120 , a correlation component synthesizing unit 130 , a gain multiplying unit 131 as a first gain multiplying unit, a first signal adding unit 132 , and a second signal adding unit 133 .

Herein, it is assumed that the audio signal processing apparatus 100 receives a stereo signal.

The first correlation component separating unit 110 receives inputs of a left channel input signal S 1 as a first signal and a right channel input signal S 2 as a second signal.

From the right channel input signal S 2 in a predetermined period, the first correlation component separating unit 110 generates a first correlation component signal S 4 having a correlation with the left channel input signal S 1 in the right channel input signal S 2 .

Further, the first correlation component separating unit 110 adds a signal of an inverted phase of the first correlation component signal S 4 to the left channel input signal S 1 to separate, from the left channel input signal S 1 , the left channel non-correlation component signal S 3 as the first non-correlation component signal having no correlation with the right channel input signal S 2 .

FIG. 2 is a block diagram schematically illustrating a configuration of the first correlation component separating unit 110 .

The first correlation component separating unit 110 includes a first predicting unit 111 and a first non-correlation component calculating unit 112 .

In the following description, the current time is referred to as time n, the time a predetermined period before time n is referred to as time n−1, the time the predetermined period before time n−1 is referred to as time n−2, . . . , and the time the predetermined period before time n−(N−1) is referred to as time n−N. Then, the right channel input signal S 2 at each of time n, time n−1, time n−2, . . . , and time n-N is represented as r(n), r(n−1), r(n−2), . . . , and r(n−N). It should be noted that N is a prediction order and is an integer of 2 or more.

The first predicting unit 111 predicts the left channel input signal S 1 based on r(n), r(n−2), . . . , r(n−N) and a prediction coefficient, treats the predicted signal as a correlation component, and supplies the correlation component as the first correlation component signal S 4 to the first non-correlation component calculating unit 112 and the correlation component synthesizing unit 130 shown in FIG. 1 . For example, the first correlation component signal S 4 is calculated by convolving r(n), r(n−2), . . . , r(n−N) and the prediction coefficient.

As the algorithm used for the prediction, for example, an LMS (Least-Mean-Square) algorithm which is a known adaptive filter technology may be used. That is, the first predicting unit 111 predicts the left channel input signal S 1 by the adaptive filter process.

When the adaptive filter technology such as the LMS algorithm is applied to the first predicting unit 111 , the first predicting unit 111 updates the value of the prediction coefficient upon receiving the left channel non-correlation component signal S 3 . This is because the left channel non-correlation component signal S 3 is an error signal indicating a prediction error in the adaptive filter technology. Therefore, the first predicting unit 111 predicts the left channel input signal S 1 by updating the value of the prediction coefficient so that the error signal approaches zero to, thereby generating the first correlation component signal S 4 including a human voice having a high correlation with the left channel input signal S 1 in the right channel input signal S 2 .

Returning to FIG. 1 , the second correlation component separating unit 120 receives inputs of the right channel input signal S 2 and the left channel input signal S 1 .

From the left channel input signal S 1 in a predetermined period, the second correlation component separating unit 120 generates a second correlation component signal S 6 having a correlation with the right channel input signal S 2 in the left channel input signal S 1 .

Further, the second correlation component separating unit 120 adds a signal of an inverted phase of the second correlation component signal S 6 to the right channel input signal S 2 to separate, from the right channel input signal S 2 , the right channel non-correlation component signal S 5 as the second non-correlation component signal having no correlation with the left channel input signal S 1 .

FIG. 3 is a block diagram schematically illustrating a configuration of the second correlation component separating unit 120 .

The second correlation component separating unit 120 includes a second predicting unit 121 and a second non-correlation component calculating unit 122 .

In the following description, the left channel input signal S 1 at each of time n, time n−1, time n−2, . . . , and time n−N is represented by 1(n), 1(n−1), 1(n−2), . . . , 1(n−N).

The second predicting unit 121 predicts the right channel input signal S 2 based on 1(n), 1(n−1), 1(n−2), . . . , 1(n−N) and a prediction coefficient, treats the predicted signal as a correlation component, and supplies the correlation component as the second correlation component signal S 6 to the second non-correlation component calculating unit 122 and the correlation component synthesizing unit 130 shown in FIG. 1 . For example, the second correlation component signal S 6 is calculated by convolving 1(n), 1(n−1), 1(n−2), . . . , 1(n−N) and the prediction coefficient.

As the algorithm used for prediction, the LMS algorithm or the like may be used in the same manner as in the first predicting unit 111 .

When an adaptive filter technology such as the LMS algorithm is applied to the second predicting unit 121 , the second predicting unit 121 updates the value of the prediction coefficient upon receiving the right channel non-correlation component signal S 5 described later. This is because the right channel non-correlation component signal S 5 is an error signal indicating a prediction error in the adaptive filter technology. Therefore, the second predicting unit 121 predicts the right channel input signal S 2 by updating the value of the prediction coefficient so that the error signal approaches zero, thereby generating the second correlation component signal S 6 including a human voice having a high correlation with the right channel input signal S 2 in the left channel input signal S 1 .

The second non-correlation component calculating unit 122 inverts the phase of the second correlation component signal S 6 supplied from the second predicting unit 121 and adds the phase-inverted second correlation component signal S 6 and the right channel input signal S 2 to calculate the right channel non-correlation component signal S 5 . As described above, the right channel non-correlation component signal S 5 is an error signal in the adaptive filter technology.

Returning to FIG. 1 , the correlation component synthesizing unit 130 receives the first correlation component signal S 4 and the second correlation component signal S 6 , and adds these two signals to synthesize them, thereby calculating a synthesized correlation component signal S 7 .

For example, the correlation component synthesizing unit 130 performs a process based on the following Equation (1) and supplies the calculated X P (n) to the gain multiplying unit 131 as a synthesized correlation component signal S 7 .

Equation (1)

x p ( n )=( l p ( n )+ r p ( n ))/2  (1)

In the above equation, l P (n) represents the first correlation component signal S 4 , and r P (n) represents the second correlation component signal S 6 .

The gain multiplying unit 131 receives the synthesized correlation component signal S 7 , multiply the synthesized correlation component signal S 7 by a gain, and supplies the synthesized correlation component signal multiplied by the gain to a first signal adding unit 132 and a second signal adding unit 133 as a correlation component signal S 8 .

Here, since the synthesized correlation component signal S 7 contains many components of human voices, the gain for the multiplication is preferably larger than 1. In addition, the value of the gain may be a fixed value or a variable value set by a user using a GUI (Graphical User Interface) via an input unit and a display unit not shown.

A first signal adding unit 132 adds the left channel non-correlation component signal S 3 and the correlation component signal S 8 to generate a left channel output signal S 9 as a final output. The left channel output signal S 9 thus generated is output to a subsequent stage of the audio signal processing apparatus 100 .

Similarly, the second signal adding unit 133 adds the right channel non-correlation component signal S 5 and the correlation component signal S 8 to generate a right channel output signal S 10 as a final output. The right channel output signal S 10 thus generated is output to a subsequent stage of the audio signal processing apparatus 100 .

The audio signal processing apparatus 100 can be implemented by hardware (H/W) or software (S/W).

FIG. 4A is a block diagram illustrating an example in which the audio signal processing apparatus 100 is implemented by H/W.

The audio signal processing apparatus 100 can be implemented by a processing circuit 150 . In this case, the processing circuit 150 receives a stereo signal from a media reproducing device 151 or a broadcast wave receiving device 152 . The stereo signal processed by the processing circuit 150 is converted into an analog signal by a DAC circuit 153 and passed to a speaker 155 via an amplifier 154 . It should be noted that the media reproducing device 151 is a device for reading digital information from a medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a BD (Blu-ray Disc).

Further, a display device 156 functions as a display unit for displaying a screen image for changing the gain value, and an input device 157 functions as an input unit for inputting the gain value.

FIG. 4B is a block diagram illustrating an example in which the audio signal processing apparatus 100 is implemented by S/W.

The audio signal processing apparatus 100 can be implemented by reading a program stored in an external storage device 160 into a memory 161 and executing the program by a processor 162 . In this case, the processor 162 processes the data stored in the external storage device 160 or the data expanded in the memory 161 . The external storage device 160 is, for example, a storage device such as a hard disk drive (HDD) or a solid state drive (SSD) connected directly or via a network.

It should be noted that the media reproducing device 151 , the broadcast wave receiving device 152 , the speaker 155 , the display device 156 , or the input device 157 may be connected.

The processing circuit 150 , the media reproducing device 151 , or the broadcast wave receiving device 152 , the DAC circuit 153 , the amplifier 154 , the speaker 155 , the display device 156 , and the input device 157 shown in FIG. 4A may constitute an audio device 100 .

Alternatively, the external storage device 160 , the memory 161 , the processor 162 , the media reproducing device 151 or the broadcast wave receiving device 152 , the speaker 155 , the display device 156 , and the input device 157 shown in FIG. 4B may constitute an audio device 100 .

FIG. 5 is a flowchart indicating a process in the audio signal processing apparatus 100 in Embodiment 1.

First, the first correlation component separating unit 110 receives the inputs of a left channel input signal S 1 and a right channel input signal S 2 , and generates a left channel non-correlation component signal S 3 and a first correlation component signal S 4 (ST 10 ).

Further, the second correlation component separating unit 120 receives the inputs of the right channel input signal S 2 and the left channel input signal S 1 and generates a right channel non-correlation component signal S 5 and a second correlation component signal S 6 (ST 11 ).

Next, the correlation component synthesizing unit 130 synthesizes the first correlation component signal S 4 and the second correlation component signal S 6 to generate a synthesized correlation component signal S 7 (ST 12 ).

Next, the gain multiplying unit 131 multiplies the synthesized correlation component signal S 7 by a gain to generate a correlation component signal S 8 (ST 13 ).

Next, the first signal adding unit 132 adds the left channel non-correlation component signal S 3 and the correlation component signal S 8 to generate a left channel output signal S 9 (ST 14 ).

The second signal adding unit 133 adds the right channel non-correlation component signal S 5 and the correlation component signal S 8 to generate a right channel output signal S 10 (ST 15 ).

As described above, according to Embodiment 1, it is possible to improve the ease of hearing human voices by separating the input signal into the correlation component signal and the non-correlation component signal by using the correlation component separating units 110 , 120 and by multiplying the correlation component signal by a gain.

Further, since the algorithm of the adaptive filter is used to extract the correlation component, it is possible to extract the correlation component shifted by several milliseconds in the left and right channels of stereo signals.

FIG. 6 is a block diagram schematically illustrating a configuration of an audio signal processing apparatus 200 according to Embodiment 2.

The audio signal processing apparatus 200 includes a first correlation component separating unit 110 , a second correlation component separating unit 120 , a correlation component synthesizing unit 130 , a gain multiplying unit 131 , a first signal adding unit 132 , a second signal adding unit 133 , and a band enhancing unit 234 .

The audio signal processing apparatus 200 according to Embodiment 2 is configured in the same manner as the audio signal processing apparatus 100 according to Embodiment 1 except that the band enhancing unit 234 is added.

It should be noted that the correlation component synthesizing unit 130 supplies the synthesized correlation component signal S 7 to the band enhancing unit 234 , and the gain multiplying unit 131 multiplies the enhanced synthesized correlation component signal S 11 supplied from the band enhancing unit 234 by a gain, as will be described later.

The band enhancing unit 234 receives the synthesized correlation component signal S 7 and enhances a band that is easy for a person to hear in the synthesized correlation component signal S 7 by filter processing. The digital filter used by the band enhancing unit 234 may be implemented by a FIR (Finite Impulse Response) filter or an IIR (Infinite Impulse Response) filter. FIG. 7 shows an example of frequency characteristics of a digital filter used for band enhancement.

The band that is easy for a person to hear is a band important for the ease of hearing a person's voice.

The band enhancing unit 234 provides the band-enhanced and synthesized correlation component signal to the gain multiplying unit 131 as an enhanced synthesized correlation component signal S 11 .

As described above, according to Embodiment 2, since the band enhancing unit 234 enhances the band which is important for the ease of hearing human voices, the clearness of the human voice is further improved.

FIG. 8 is a block diagram schematically illustrating a configuration of an audio signal processing apparatus 300 according to Embodiment 3.

The audio signal processing apparatus 300 includes a first correlation component separating unit 110 , a second correlation component separating unit 120 , a correlation component synthesizing unit 130 , a gain multiplying unit 131 , a first signal adding unit 132 , a second signal adding unit 133 , a band enhancing unit 234 , a gain multiplying unit 335 as a second gain multiplying unit, and a gain multiplying unit 336 as a third gain multiplying unit.

The audio signal processing apparatus 300 according to Embodiment 3 is configured in the same manner as the audio signal processing apparatus 200 according to Embodiment 2, except that the gain multiplying unit 335 and the gain multiplying unit 336 are added.

It should be noted that the first correlation component separating unit 110 supplies the separated left channel non-correlation component signal S 3 to the gain multiplying unit 335 , and the second correlation component separating unit 120 supplies the separated right channel non-correlation component signal S 5 to the gain multiplying unit 336 .

In addition, the first signal adding unit 132 adds the multiplied left channel non-correlation component signal S 12 supplied from the gain multiplying unit 335 and the correlation component signal S 8 , and the second signal adding unit 133 adds the multiplied right channel non-correlation component signal S 13 supplied from the gain multiplying unit 336 and the correlation component signal S 8 .

The gain multiplying unit 335 receives the left channel non-correlation component signal S 3 , multiplies the left channel non-correlation component signal S 3 by a gain, and supplies the gain-multiplied left channel non-correlation component signal to the first signal adding unit 132 as the multiplied left channel non-correlation component signal S 12 . Here, since the left channel non-correlation component signal S 3 mainly contains components other than the human voice, the gain for the multiplication is desirably smaller than 1. Also, the gain value may be a fixed value or a variable value set by a user using a GUI as described above.

The gain multiplying unit 336 receives the right channel non-correlation component signal S 5 , multiplies the right channel non-correlation component signal S 5 by a gain, and supplies the gain-multiplied right channel non-correlation component signal to the second signal adding unit 133 as the multiplied right channel non-correlation component signal S 13 . Here, since the right channel non-correlation component signal S 5 mainly contains components other than the human voice, the gain of multiplication is desirably smaller than 1. Also, the gain value may be a fixed value or a variable value set by a user using a GUI as described above.

As described above, according to Embodiment 3, since the gain multiplying units 335 , 336 can reduce the volume of components other than the human voice, the clearness of the human voice is further improved.

In Embodiment 3, the band enhancing unit 234 may not be provided.

100 , 200 , 300 audio signal processing apparatus, 110 first correlation component separating unit, 111 first predicting unit, 112 first non-correlation component calculating unit, 120 second correlation component separating unit, 121 second predicting unit, 122 second non-correlation component calculating unit, 130 correlation component synthesizing unit, 131 gain multiplying unit, 132 first signal adding unit, 133 second signal adding unit, 234 band enhancing unit, 335 gain multiplying unit, 336 gain multiplying unit