Method for separating audio sources and audio system using the same

The present application claims the benefit under 35 U.S.C. §119(a) to a Korean patent application filed in the Korean Intellectual Property Office on Jun. 11, 2014, and assigned Serial No. 10-2014-0070876, the entire disclosure of which is hereby incorporated by reference.

The present invention relates generally to a method for separating audio sources, and more particularly, to a method for separating audio sources from a mixed audio signal, and an audio system using the same.

FIG. 1 illustrates a view showing the concept of a related-art method for separating audio sources. In FIG. 1 , s 1 , s 2 , and s 3 are three (3) different audio sources, and x is a mixed audio signal, That is, x is a mix signal of s 1 , s 2 , and s 3 .

As shown in FIG. 1 , there is no overlap among the audio sources s 1 , s 2 , and s 3 . That is, the audio sources s 1 , s 2 , and s 3 are independent of one another.

In this circumstance, there is no problem in separating the audio signal x into the audio sources s 1 , s 2 , and s 3 . This is because an audio component constituting the audio signal x can be matched with one of the audio sources s 1 , s 2 , and s 3 .

However, the audio signal x and the audio sources s 1 , s 2 , and s 3 shown in FIG. 1 are the ideal or very special case. In practice, the audio signal x and the audio sources s 1 , s 2 , and s 3 are in the state shown in FIG. 2 .

That is, the audio sources s 1 , s 2 , and s 3 are not completely independent of one another. That is, there is an overlap among the audio sources s 1 , s 2 , and s 3 . In this circumstance, there is no problem in mixing the audio sources s 1 , s 2 , and s 3 into the single audio signal x.

However, a problem arises when the mixed audio signal x is separated into the audio sources s 1 , s 2 , and s 3 . This is because an audio component corresponding to the overlapping area of the audio sources s 1 , s 2 , and s 3 cannot be matched with one of the audio sources s 1 , s 2 , and s 3 .

Due to this problem, an audio source separation algorithm processes the audio signal x and the audio sources s 1 , s 2 , and s 3 on the assumption that the audio signal x and the audio sources s 1 , s 2 , and s 3 are in the state shown in FIG. 1 even if the audio signal x and the audio sources s 1 , s 2 , and s 3 are actually in the state shown in FIG. 2 .

Since the audio sources are separated without considering the real state of the audio signal and the audio sources, excellent audio source separation performance would not be guaranteed and it is.

To address the above-discussed deficiencies of the prior art, it is a primary aspect of the present invention to provide a method for separating audio sources, which is based on a method for separating an audio signal corresponding to at least two of audio sources as a residual signal in separating audio sources from a mixed audio signal, and an audio system using the same.

According to one aspect of the present invention, a method for separating audio sources includes: receiving a mixed audio signal; and a first separation operation of separating the input mixed audio signal into a plurality of audio sources and a first residual signal.

The first residual signal may be an audio signal which is common to at least two of the plurality of audio sources.

The method may further include: a second separation operation of separating the residual signal separated by the first separation operation into residual signals corresponding to the plurality of audio sources and a second residual signal; and adding the residual signals to the audio sources, respectively.

The first separation operation and the second separation operation may be performed by using a Nonnegative Matrix Factorization-Expectation Maximization (NMF-EM) method, and the second separation operation may use parameters which are determined based on initial parameters used in the first separation operation and parameters updated by the first separation operation.

The second separation operation may use parameters which are obtained by giving weightings to the determined parameters.

The weighting may be determined based on an absolute power average of the mixed audio signal and an absolute power average of the first residual signal.

According to another aspect of the present invention, an audio system includes: an input unit configured to receive a mixed audio signal; and a separation unit configured to separate the input mixed audio signal into a plurality of audio sources and a first residual signal.

As described above, according to exemplary embodiments of the present invention, the concept of a residual signal is introduced to separate a mixed audio signal into audio sources, and an audio signal corresponding to at least two of the audio sources is separated as a residual signal. Therefore, audio separation performance can be improved.

In addition, according to exemplary embodiments of the present invention, a separated residual signal may be re-separated and separated residual signals may be added to corresponding audio sources. Therefore, audio sources can be separated more completely.

For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which like reference numerals represent like parts:

FIG. 1 is a view showing the concept of a related-art method for separating audio sources;

FIG. 2 is a view showing a relationship between a real audio signal and audio sources;

FIG. 3 is a block diagram of an audio system according to an exemplary embodiment of the present invention; and

FIGS. 4 to 7 are graphs showing results of evaluating audio separation performance.

Reference will now be made in detail to the embodiment of the present general inventive concept, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiment is described below in order to explain the present general inventive concept by referring to the drawings.

FIG. 3 is a block diagram of an audio system according to an exemplary embodiment of the present invention. The audio system according to an exemplary embodiment of the present invention is a system for separating an audio signal into audio sources.

The audio system performing the above-mentioned function includes an audio signal separation unit 110 , a parameter update unit 120 , a residual signal separation unit 130 , and an audio source combination unit 140 as shown in FIG. 3 .

In an exemplary embodiment, it is assumed that an audio signal x is a signal in which J number of audio sources (objects) s 0 , . . . , s J-1 are mixed.

The audio signal separation unit 110 separates the input audio signal x into a plurality of audio sources s′ 0 , . . . , s′ J-1 and a residual signal r 1 . The residual signal r 1 corresponds to an audio signal which is common to at least two of the audio sources s 0 , . . . , s J-1 (overlapping area).

Since the residual signal r 1 is separated from the audio signal x, the audio sources s′ 0 , . . . , s′ J-1 separated from the audio signal x by the audio signal separation unit 110 are different from the original audio sources s 0 , . . . , s J-1 which are the base for mixing the audio signal x.

The audio signal separation unit 110 uses a Nonnegative Matrix Factorization-Expectation Maximization (NMF-EM) method to separate the audio signal x.

The NMF-EM method is a well-known audio separation method and thus a detailed description thereof is omitted here.

In the related-art method using the NMF-EM method to separate the audio signal, updated parameters {W u ′H u ′} are generated from initial parameters {W′H′} regarding the audio sources, and audio sources are determined according to the updated parameters {W u ′H u ′}.

However, in the exemplary embodiment of the present invention, since the residual signal r 1 is separated from the audio signal in addition to the audio sources, it should be noted that the initial parameters {W′H′} and the updated parameters {W u ′H u ′} further include a parameter regarding the residual signal r 1 in addition to the parameters regarding the audio sources.

The residual signal separation unit 130 re-separates the residual signal r 1 separated by the audio signal separation unit 110 . Specifically, the residual signal separation unit 130 separates the residual signal r 1 into residual signals r 1,s0 , . . . , r 1,sJ-1 regarding the audio sources and a residual signal r 2 .

The residual signal r 2 is a signal that cannot be included in the residual signals r 1,s0 , . . . , r 1,sJ-1 regarding the audio sources. Conceptually, the residual signal r 2 may be interpreted as the residual signal r 1 which is common to the at least two of the audio sources s 0 , . . . , s J-1 (overlapping area).

The residual signal separation unit 130 separates the residual signal r 1 by using the NMF-EM method. However, initial parameters {W n ′H n ′} used in the NMF-EM method are calculated by the parameter update unit 120 according to following Equation 1:

{ W′ n W′ n }=w 2 ×[w 1 {W′H′}+ (1− w 1 ) {W′ u H′ u }]  Equation 1

where {W′H′} indicates initial parameters which are used by the audio signal separation unit 110 to separate the audio signal x, and {W′ u H′ u } indicate parameters which are updated during the audio separation process of the audio signal separation unit 110 .

Parameters used to separate the residual signal r 1 are obtained based on a sum of weightings given to the initial parameters used to separate the audio signal x and weightings given to the updated parameters which are generated as a result of the separating.

The weighting w 1 is to determine weights of the initial parameters {W′H′} and the updated parameters {W′ u H′ u } and satisfies 0≦w 1 ≦1. The weighting w 2 is to determine weights of the initial parameters {W′H′} and the updated parameters {W′ u H′ u } and satisfies 0≦w 2 ≦1.

The weighting w 2 is determined based on a ratio between an absolute power average of the audio signal x and an absolute power average of the residual signal r 1 , and is expressed by following Equation 2:

The audio source combination unit 140 generates final audio sources by adding the residual signals r 1,s0 , . . . , r 1,sJ-1 regarding the audio sources separated by the residual signal separation unit 130 to the audio sources s′ 0 , . . . , s′ J-1 separated by the audio signal separation unit 110 .

The residual signal r 2 separated by the residual signal separation unit 130 may be discarded or may be re-separated. Specifically, the audio source combination unit 140 applies the residual signal r 2 to the residual signal separation unit 130 such that the residual signal r 2 is separated by the residual signal separation unit 130 like the residual signal r 1 .

In this case, the audio source combination unit 140 adds residual signals r 2,s0 , . . . , r 2,sJ-1 regarding the audio sources separated from the residual signal r 2 to the final audio sources. In addition, a residual signal r 3 is separated from the residual signal r 2 by the residual signal separation unit 130 .

Thereafter, it is possible to re-separate the residual signal r 3 . It is determined whether to re-separate the residual signal based on the residual signal and parameters of the audio sources.

In the exemplary embodiment described up to now, the concept of a residual signal has been introduced and the method for separating audio sources from a mixed audio signal by separating an audio signal corresponding to at least two of the audio sources as a residual signal has been described.

The method for separating audio sources described above can be applied to a monitoring system and may be used to extract only a specific audio source (e.g., a voice) from an audio signal or remove a specific audio source (e.g., a sound of a wind, a vehicle horn sound). Furthermore, this method can be applied to give an audio effect for each audio source or create contents.

FIGS. 4 to 7 illustrate results of evaluating audio separation performance. As shown in FIGS. 4 to 7 , the audio source separation performance achieved by using the residual signal is better than the performance that does not use the residual signal. In addition, the performance can be enhanced when the residual signal separation method is applied.

Although the present disclosure has been described with an exemplary embodiment, various changes and modifications may be suggested to one skilled in the art. It is intended that the present disclosure encompass such changes and modifications as fall within the scope of the appended claims.