Signal separating apparatus, signal separating method, signal separating program and recording medium

It is assumed here that the number N of signal sources is 3 and the squares of the distances between the normalized basis vectors A 1 ″ (f), A 2 ″(f), and A 3 ″(f) at an frequency f and the centroids η 1 , η 2 , and η 3 are as shown in the following table.

Here, the permutation obtained according to Equation (13) is

Π f : [1,2,3]→[2,3,1]

because the combinations

 η 1 - A Π ⁡ ( 1 ) ″ ⁡ ( f )  2 =  η 1 - A 2 ″ ⁡ ( f )  2 = 0.1  η 2 - A Π ⁡ ( 2 ) ″ ⁡ ( f )  2 =  η 2 - A 3 ″ ⁡ ( f )  2 = 0.2  η 3 - A Π ⁡ ( 3 ) ″ ⁡ ( f )  2 =  η 3 - A 1 ″ ⁡ ( f )  2 = 0.15

minimize

[ Formula ⁢ ⁢ 17 ] ∑ k = 1 3 ⁢  η k - A Π ⁡ ( k ) ″ ⁡ ( f )  2

(End of the Description of Example 1 of Determination of Permutation Π f )

However, this procedure will be unrealistic if N is large. Therefore, an approximation method may be used in which A Π(k) ″(f) that minimize ∥η k −A Π(k) ″(f)∥ 2 are selected one by one in such a manner that there are no overlaps and a permutation that transfers the selected A Π(k) ″(f) to the normalized basis vector A k ″ (f) is chosen as the permutation Π f . A procedure for determining the permutation Π f using this approximation method under the same conditions given in Example 1 of determination of permutation Π f will be described below.

First, because the minimum square of distance in Table 1 is 0.1 (the square of the distance between the normalized basis vector A 2 ″(f) and centroid η 1 ), Π(1)=2 is chosen. Then, the row and column relating to the normalized basis vector A 2 ″(f) and centroid η 1 are deleted as shown below.

Because the minimum square of distance in Table 2 is 0.15 (the square of the distance between the normalized basis vector A 1 ″(f) and centroid η 3 ), Π(3)=1 is chosen. Finally, the remainder, 3 is assigned to Π(2). (End of the description of Example 2 of determination of permutation Π f )

Then, the sorting section 145 reads the first separation matrix W(f) from memory area 103 of the memory 100 and the permutation Π f from memory area 110 . The sorting section 145 rearranges the rows of the first separation matrix W(f) in accordance with the permutation Π f to generate a second separation matrix W′(f) and stores it in memory 111 of the memory 100 (step S 7 ). The rearrangement of the first separation matrix W(f) according to the permutation Π f means that rearrangement equivalent to the rearrangement of the elements A Π(k) ″(f) to the elements A k ″(f) in the Moore-Penrose generalized inverse W + (f) described above is performed on the first separation matrix W(f). That is, the first separation matrix W(f) is rearranged in such a manner that the Π f (k)-th row of the first separation matrix W(f) becomes the k-th row of the second separation matrix W′(f). In the Examples 1 and 2 of determination of permutation Π f , the second, third, and first rows of the first separation matrix W(f) become the first, second, and third rows, respectively, of the second separation matrix W′(f).

[Processing by the Separated Signal Generating Section 150 ]

Then, the separated signal generating section 150 reads the mixed signals X q (f, τ) in the frequency domain from memory 102 of the memory 100 and the second separation matrix W′(f) from memory area 111 . The separated signal generating section 150 then uses the mixed-signal vector X(f, τ)=[X 1 (f, τ), . . . , X M (f, τ)] T consisting of the mixed signals X q (f, τ) in the frequency domain and the second separation matrix W′(f) to calculate a separated signal vector

Y ( f ,τ)= W ′( f )· X ( f ,τ)

and stores the frequency-domain signals Y p (f, τ) which are the elements of the separated signal vector (which are referred to as “frequency-domain mixed signals) in memory area 112 of the memory 100 (step S 8 ).

[Processing by the Time Domain Transforming Section 160 ]

Finally, the time domain transforming section 160 reads the frequency-domain separated signals Y p (f, τ) from memory 112 of the memory 100 , transforms them into separated signals y p (t) in the time domain one by one for each suffix p (for each Y p (f, τ)) by using transformation such as short-time inverse Fourier transformation, and stores the separated signals y p (t) in the time domain in memory area 113 of the memory 110 (step S 9 ).

[Details of Normalization (Details of Step S 4 )]

Details of the above-mentioned normalization (step S 4 ) performed by the basis vector normalizing section 142 will be described below.

FIG. 6 is a flowchart illustrating details of the normalization process.

First, the control section 170 ( FIG. 3 ) assign 1 to parameter p and stores it in the temporary memory 171 (step S 11 ). The control section 170 also assigns 1 to parameter q and stores it in the temporary memory 171 (step S 12 ). Then, the frequency normalizing section 142 a ( FIG. 4 ) reads the parameters d, c, and Q described above from memory area 107 of the memory 100 , reads parameters p and q from the temporary memory 171 , and, for the elements A qp (f) of the basis vector A p (f), calculates

[Formula  18] A qp ′ ⁡ ( f ) =  A qp ⁡ ( f )  ⁢ exp ⁡ [ j ⁢ arg ⁡ [ A qp ⁡ ( f ) / A Qp ⁡ ( f ) ] 4 ⁢ fc - 1 ⁢ d ] ( 14 )

then, stores the calculated A qp ′(f) in memory area 105 of the memory 100 as the elements A qp ′(f) of the frequency-normalized vector A p ′(f) (step S 13 ). Here, arg[·] represents the argument of · and j is the imaginary unit.

In particular, the first normalizing section 142 aa of the frequency normalizing section 142 a first normalizes the argument of each element A qp (f) of a basis vector A p (f) on the basis of a particular element A Qp (f) of the basis vector A p (f) by

[Formula 19]

A qp ′″( f )=| A qp ( f )|exp{ j ·arg[ A qp ( f )/ A QP ( f )]}  (15)

Then, the second normalizing section 142 ab of the frequency normalizing section 142 a divides the argument of each of the elements A qp ′″(f) normalized by the first normalizing section 142 aa by a value 4fc −1 d proportional to the frequency f as

Then, the control section 170 determines whether the value of parameter q stored in the temporary memory 171 satisfies q=M (step S 14 ). If not q=M, the control section 170 sets a calculation result q+1 as a new value of the parameter q, stores it in the temporary memory 171 (step S 15 ), and returns to step S 13 . On the other hand, if q=M, then the control section 170 determines whether p=N (step S 16 ).

If not p=N, then the control section 170 sets a calculation result p+1 as a new value of the parameter p, stores it in the temporary memory 171 (step S 17 ), and then returns to step S 12 . On the other hand, if p=N, the control section 170 assigns 1 to the parameter p, and stores it in the temporary memory 171 (step S 18 ). Then the norm normalizing section 142 b starts processing. The norm normalizing section 142 b first reads the elements A qp ′(f) of the frequency-normalized vector A p ′(f) from memory area 105 of the memory 100 , calculates

[ Formula ⁢ ⁢ 21 ]  A p ′ ⁡ ( f )  = ∑ q = 1 M ⁢ ( A pq ′ ⁡ ( f ) ) 2 ( 17 )

to obtain the norm ∥A p ′(f)∥ of the frequency-normalized vector A p ′(f), and stores the frequency-normalized vector A p ′(f) and its norm ∥A p ′(f)∥ in the temporary memory 171 (step S 19 ).

Then, the norm normalizing section 142 b reads the frequency-normalized vector A p ′(f) and its norm ∥A p ′(f)∥ from the temporary memory 171 , calculates

A p ″( f )= A p ′( f )/∥ A p ′( f )∥  (18)

to obtain a normalized basis vector A p ″(f), and stores it in memory area 106 of the memory 100 (step S 20 ).

Then, the control section 170 determines whether the value of parameter p stored in the temporary memory 171 satisfies p=N (step S 21 ). If not p=N, the control section 170 sets a calculation result p+1 as a new value of the parameter p, stores it in the temporary memory 171 (step S 22 ), and then returns to step S 19 . On the other hand, if p=N, the control section 170 terminates the processing at step S 4 .

The normalized basis vectors A p ″(f) thus generated are not dependent on frequency and dependent only on the positions of the signal sources. Consequently, the normalized basis vectors A p ″(f) forms clusters. The reason will be described below.

[Reason Whey Normalized Basis Vectors A p ″(f) Form Clusters]

Each of the elements A qp (f) of a basis vector A p (f) is proportional to the frequency response H qk from the signal source k corresponding to a source signal p to a sensor q (that is, it is equal to the frequency response multiplied by a complex scalar). These complex scalars change with discrete time whereas the relative value between the complex scalar corresponding to the source signal p and sensor q and the complex scalar corresponding to the source signal p and sensor Q does not change with changing discrete time (provided that the frequency f is the same). That is, if the frequency f is the same, the relative value between the argument of the complex scalar corresponding to the source signal p and sensor q and the argument of the complex scalar corresponding to the source signal p and sensor Q is constant.

As described above, the first normalizing section 142 aa of the frequency normalizing section 142 a normalizes the argument of each element A qp (F) of a basis vector A p (f) on the basis of one particular element A Qp (f) of that basis vector A p (f). Thus, uncertainty due to the phase of the complex scalars mentioned above is eliminated and the argument of the element A qp (f) of the basis vector A p (f) corresponding to the source signal p and sensor q is represented as a value relative to the argument of the element A Qp (F) of the basis vector A p (f) corresponding to the source signal p and sensor Q (corresponding to the reference value Q). The relative value corresponding to the argument of the element A Qp (f) is represented as 0. The frequency response from a signal source k to a sensor q is approximated using a direct-wave model without reflections and reverberations. Then the argument normalized by the first normalizing section 142 aa is proportional to both of the arrival time difference of waves from the signal source k to the sensor and the frequency f. The arrival time difference here is the difference between the time taken for a wave from the signal source k to reach the sensor q and the time taken for the wave to reach the reference sensor Q.

As has been describe above, the second normalizing section 142 ab divides the argument of each element A qp ′″(f) normalized by the first normalizing section 142 aa by a value proportional to the frequency f. Thus, the elements A qp ′″(f) are normalized to elements A qp ′(f) excluding dependence of their arguments on frequency. Consequently, according to the direct-wave model, each of the normalized elements A qp ′(f) depends only on the arrival time difference between the times at which the wave from the signal source k reaches the sensors. The arrival time difference of the wave from the signal source k to the sensors depends only on the relative positions of the signal source k, sensor q, and reference sensor Q. Accordingly, the arguments of the elements A qp ′(f) with the same signal source k, sensor q, and reference sensor Q are the same even if the frequency varies. Thus, the frequency-normalized vectors A p ′(f) are not dependent on the frequency f but only on the positions of signal source k.

Therefore, by clustering the normalized basis vectors A p ″ (f) resulting from normalization of the norms of the frequency-normalized vectors A p ′(f), clusters are generated, each of which corresponds to the same signal source. Although the direct-wave model is not exactly satisfied in a real environment because of reflections and reverberations, a sufficiently good approximation can be obtained as shown in experimental results which will be given later.

The reason why the normalized basis vectors A p ″(f) forms clusters will be described below with respect to a model. The impulse response h qk (r) in Equation (1) described earlier is approximated using a direct-wave (near-field) mixture model and represented in the frequency domain as

[Formula  22] H qk ⁡ ( f ) = 1 d qk ⁢ exp ⁡ [ - j2π ⁢ ⁢ fc - 1 ⁡ ( d qk - d Qk ) ] ( 19 )

where d qk is the distance between a signal source k and a sensor q. The attenuation 1/d qk is determined by the distance d qk and the delay (d qk −d Qk )/c is determined by the distance normalized at the position of the reference sensor Q.

If order ambiguity and scaling ambiguity in independent component analysis (ICA) are taken into consideration, the following relation holds between the basis vector A p (f) and the vector H k (f) consisting of frequency responses from the signal source k to the sensors.

A p ( f )=ε p ·H k ( f ), A qp ( f )=ε p ·H qk ( f )  (20)

where ε p is a complex scalar value representing the ambiguity of the scaling. The possibility that suffixes k and p differ from each other represents the ambiguity of the order. From Equations (16), (18), (19), and (20),

[Formula  23] A qp ″ ⁡ ( f ) = 1 d qk ⁢ D ⁢ exp ⁡ [ - j ⁢ ⁢ π 2 ⁢ ( d qk - d Qk ) d ] , D = ∑ i = 1 M ⁢ 1 d ik 2 ( 21 )

As can be seen from this equation, the elements A qp ″(f) of the normalized basis vector A p ″(f) are independent of the frequency f and dependent only on the positions of the signal sources k and sensors q. Therefore, clustering the normalized basis vectors A p ″(f) generates clusters, each corresponding to the same signal source.

The same applies to a near-field mixture model in which signal attenuation is not taken into consideration. The convolutive mixture model represented by Equation (1) given earlier is approximated with a near-field mixture model in which attenuation is ignored and represented in the frequency domain as

[Formula 24]

H qk ( f )=exp[− j 2 πfc −1 ( d qk −d Qk )]  (22)

From Equations (16), (18), (20), and (22), it follows that

[Formula  25] A qp ′′ ⁡ ( f ) = 1 M ⁢ exp ⁡ [ - j ⁢ ⁢ π 2 ⁢ ( d qk - d Qk ) d ] ( 23 )

Again, the elements A qp ″(f) of the normalized basis vector A p ″(f) are independent of the frequency f and dependent only on the positions of the signal source k and sensor q.

Also, the same applies to a far-field mixture model as well as the near-field mixture model. The convolutive mixture model represented by Equation 1 mentioned above is approximated and represented in the frequency domain as

[Formula 26]

H qk ( f )=exp[− j 2π fc −1 ∥SE q −SE Q ∥cos θ k qQ ]  (24)

Here, SE q and SE Q are vectors representing the positions of sensors q and Q, and θ k qQ is the angle between the straight line connecting sensors q and Q and the straight line connecting the center points of sensors q and Q and the signal source k. From Equations (16), (18), (20), and (24),

[Formula  27] A qp ″ ⁡ ( f ) = 1 M ⁢ exp [ - j ⁢ ⁢ π 2 ⁢  SE q - SE Q  ⁢ cos ⁢ ⁢ θ k qQ d ] ( 25 )

Again, the elements A qp ″(f) of the normalized basis vector A p ″(f) are independent of the frequency f and dependent only on the positions of the signal source k and sensor q.

Preferably, the value of the parameter d is d>d max /2 (where d max represents the maximum distance between the reference sensor Q corresponding to element A Qp (f) and another sensor) from Equation (21), more preferably, d≧d max , and more preferably, d=d max . The reason will be described below.

FIGS. 7 and 8 are complex planes for illustrating the relation between an element A qp ″(f) of a normalized basis vector and its argument arg[A qp ″(f)] at each value of parameter d. The horizontal axis in FIGS. 7 and 8 represents the real axis and the vertical axis represents the imaginary axis.

FIG. 7A is a complex plane in the case where d max /2≧d. From the definition of d max given above, the absolute value of d qk −d Qk for any q and k is less than or equal to d max . Therefore, if d max /2≧d, then (π/2)·(d qk −d Qk )/d≦−π and (π/2)·(d qk −d Qk )/d≧π. Consequently, the arguments arg[A qp ″ (f)] of A qp ″(f) represented by Equation (21) can be distributed over the range beyond 2π, α 1 ≦arg[A qp ″(f)]≦α 2 (α 1 ≦−π and α2>). Accordingly, the arguments of elements A qp ″(f) of different normalized basis vectors can match and consequently the different normalized basis vectors A p ″(f) can be clustered into the same cluster in the clustering described above. Therefore, it is desirable that d>d max /2. However, if there is not a sample of the normalized basis vector A p ″(f) that falls in an argument overlapping range, no problem arises even if d max /2>d.

FIG. 7B shows a complex plane in the case where d max /2<d<d max . In this case, the relations −π<(π/2)·(d qk −d Qk )/d<−π/2 and π/2<(π/2)·(d qk −d Qk )/d<π can hold. Consequently, the arguments arg[A qp ″(f)] of A qp ″(f) represented by Equation (21) can be distributed over the range β 1 ≦arg[A qp ″(f)]≦β 2 (−π<β 1 <−π/2 and π/2<β 2 <π). Therefore, in the ranges −π<arg[A qp ″(f)]<−π/2 and π/2<arg[A qp ″(f)]<π, it is possible that the distance between different frequency-normalized vector elements does not monotonically increases with increasing difference between the arguments between the different frequency-normalized vector elements. This condition can degrade the accuracy of the clustering described above. Therefore, it desirable that d≧d max .

FIG. 8A is a complex plane in the case where d=d max . FIG. 8B is a complex plane in the case where d>d max . If d>d max , the relation −π/2(π/2)·(d qk −d Qk )/d<0 and 0<(π/2)·(d qk −d Qk )/d<π/2 can hold. Consequently, the arguments arg[A qp ″(f)] of A qp ″(f) represented by Equation (21) are distributed over the range γ 1 ≦arg[A qp ″(f)]≦γ 2 (−π/2<γ 1 <0 and 0<γ 2 <π/2) as shown in FIG. 8B . As d increases, the distribution range narrows and clusters are distributed more densely in the narrowed range. As a result, the accuracy of the clustering described above degrades.

In contrast, when d=d max , the relations −π/2≦(π/2)·(d qk −d Qk )/d<0 and 0<(π/2)·(d qk −d Qk )/d<π/2 can hold. Consequently, the arguments arg[A qp ″ (f)] of A qp ″ (f) represented by Equation (21) are distributed over the range −π/2≦arg[A qp ″(f)]≦π/2 as shown in FIG. 8A . In this case the cluster can be distributed in a range as wide as possible while keeping the relation in which the distance between the frequency-normalized vector elements monotonically increases with increasing difference between the arguments of the elements. As a result, typically the accuracy of the clustering can be improved.

Second Embodiment (Example of the First Aspect of the Invention)

The second embodiment of the present invention will be described below.

In the first embodiment, the permutation problem has been solved by using information obtained from basis vectors. In the second embodiment, the permutation problem is solved more accurately by combining this information with information about envelopes of separated signals as described in Japanese Patent Application Laid-Open No. 2004-145172 and H. Sawada, R. Mukai, S. Araki, S. Makino, “A Robust and Precise Method for Solving the Permutation Problem of Frequency-Domain Blind Source Separation,” IEEE Trans. Speech and Audio processing, Vol. 12, No. 5, pp. 530-538, September 2004 (hereinafter referred to as the “Reference literatures”). In these literatures, information about the directions of signal sources is used in stead of basis vectors.

The following description focuses on differences from the first embodiment and description of the same elements as those in the first embodiment will be omitted.

<Configuration>

FIG. 9 is a block diagram showing an example of a signal separating apparatus 200 according to the second embodiment. Like in the first embodiment, the signal separating apparatus 200 in the second embodiment is configured when a signal separating program is read into a CPU 10 a ( FIG. 2 ). FIG. 10A is a block diagram showing details of the permutation problem solving section 240 shown in FIG. 9 and FIG. 10B is a block diagram showing details of the permutation correcting section 247 shown in FIG. 10A . In FIGS. 9 and 10 , the same elements as those in the first embodiment are labeled with the same reference numerals as those used in the first embodiment. The dashed arrows in FIGS. 9 and 10 represent theoretical information flows whereas the solid arrows represent actual data flows. Arrows representing flows of data inputted into and outputted from a control section 170 are omitted from FIGS. 9 and 10 . Arrows representing actual data flows are also omitted from FIG. 10 .

A major difference of the second embodiment from the first embodiment lies in the configuration of the permutation problem solving section 240 . The permutation problem solving section 240 in the second embodiment is the same as the permutation problem solving section 140 in the first embodiment, except that a permutation evaluating section 246 and a permutation correcting section 247 are added in the second embodiment ( FIGS. 9 and 10A ). The permutation evaluating section 246 evaluates the reliability of a permutation on a frequency-by-frequency basis. If the reliability of a permutation at a frequency is evaluated as low, the permutation correcting section 247 calculates another permutation by using the envelope of separated signals. The permutation correcting section 247 includes a determining section 247 a , a separated signal generating section 247 b , an envelope computing section 247 c , a permutation recomputing section 247 d , and a resorting section 247 e ( FIG. 10B ). In the second embodiment, the permutation computing section 144 and the permutation correcting section 247 make up a “permutation computing section” as set fourth in claim 4 .

<Processing>

FIG. 11 is a flowchart outlining a whole process performed in the signal separating apparatus 200 according to the second embodiment. The process performed in the signal separating apparatus 200 in the second embodiment will be described with reference to the flowchart.

Steps S 51 to S 57 are the same as steps S 1 to S 7 in the first embodiment and therefore the description thereof will be omitted. In the second embodiment, after step S 57 , the reliability of a permutation Π f for each frequency is evaluated in the permutation evaluating section 246 . For a frequency for which the reliability of the permutation Π f is evaluated as low, the envelope of separated signals is used to calculate another permutation Π f ′, rows of a second separation matrix W′(f), only for that frequency are rearranged in accordance with the permutation Π f ′ to generate a third separation matrix W″ (f), and the third separation matrix W″ (f) is stored in memory area 110 of a memory 100 (step S 58 ). The processing will be detailed later.

Then, a separated signal generating section 150 reads mixed signals X q (f, τ) in the frequency domain from memory area 102 of the memory 100 and the third separation matrix W″(f) from memory 111 . The separated signal generating section 150 then uses a mixed-signal vector X(f, τ)=[X 1 (f, τ), . . . , X M (f, τ)] T consisting of the frequency-domain mixed signals X q (f, τ) and the third separation matrix W″(f) to compute a separated signal vector

Y ( f ,τ)= W ″( f )· X ( f ,τ)

and stores frequency-domain separated signals Y p (f, τ) in memory area 112 of the memory 100 (step S 59 ).

Finally, the time domain transforming section 160 reads the frequency-domain separated signals Y p (f,τ) from memory area 112 of the memory 100 , transforms them into separated signals y p (t) in the time domain for each individual suffix p, and stores the time-domain separated signals y p (t) in memory area 113 of the memory 100 (step S 60 ).

[Details of Processing at Step S 58 ]

FIGS. 12 and 13 show a flowchart illustrating an example of processing at step S 58 in FIG. 11 . Step S 58 will be detailed with reference to the flowchart.

First, a control section 170 assigns 0 to parameter f, makes a set F an empty set, and stores information representing this in a temporary memory 171 (step S 71 ). Then, the permutation evaluating section 246 evaluates the reliability of a permutation Π f stored in memory area 110 of the memory 100 for each frequency and stores the result of evaluation trust(f) in the temporary memory 171 (step S 72 ). The reliability of a permutation Π f is said to be high if the normalized basis vector A p ″(f) is sufficiently close to its corresponding centroid η k . Whether a normalized basis vector A p ″(f) is sufficiently close to its corresponding centroid η k can be determined on the basis of whether the distance between the normalized basis vector A p ″(f) and the centroid η k is smaller than the variance U k /|C k | of clusters C k :

U k /|C k |>∥η k −A Π(k) ″( f )∥ 2   (26)

At step S 72 , the permutation evaluating section 246 first reads the normalized basis vector A p ″(f) from memory area 105 of the memory 100 , the centroid η k from memory area 109 , and the permutation Π f from memory area 110 . The permutation evaluating section 246 then determines for each frequency f whether Equation 26 is satisfied. If it is satisfied, the permutation evaluating section 246 outputs and stores trust(f)=1 in the temporary memory 171 ; otherwise it outputs and stores trust(f)=0 in the temporary memory 171 .

Then, the determining section 247 a in the permutation correcting section 247 reads the evaluation result trust(f) for each frequency f from the temporary memory 171 and determines whether trust(f)=1 (step S 73 ). If trust(f)=0, the process proceeds to step S 76 . On the other hand, if trust(f)=1, the control section 170 stores the sum of sets F and {f} in the temporary memory 171 as a new set F (step S 74 ), and the re-sorting section 247 e stores the second separation matrix W′(f) at the frequency f in memory area 111 of the memory 100 as a third separation matrix W″(f) (step S 75 ), and then the proceeds to step S 76 .

At step S 76 , the control section 170 determines whether the value of parameter f stored in the temporary memory 171 satisfies the condition f=(L−1)f s /L (step S 76 ). If it does not satisfy the condition, the control section 170 stores a calculation result f+f s /L as a new value of parameter f in the temporary memory 171 (step S 77 ), and then returns to step S 72 .

On the other hand, if the value of parameter f satisfies the condition f=(L−1)f s /L, the separated signal generating section 247 b selects one frequency f that does not belong to set F. For this frequency f and the frequencies g (where gεF and |g−f|≦δ, and δ is a constant) that are in the vicinity of the frequency f and belong to set F, the separated signal generating section 247 b reads mixed signals X(f, τ)=[X 1 (f, τ), . . . , X M (f, τ)] T and X(g, τ)=[X 1 (g, τ), . . . , X M (g, τ)] T in the frequency domain from memory area 102 of the memory 100 , reads the second separation matrixes W′(f) and W′(g) from memory area 111 , and use

Y ( f ,τ)= W ′( f )· X ( f ,τ)

Y ( g ,τ)= W ′( g )· X ( g ,τ)

to compute separated signals Y(f, τ)=[Y 1 (f, τ), . . . , Y N (f, τ)] T and Y(g, τ)=[Y 1( g , τ), . . . , Y N (g, τ)] T , then stores them in the temporary memory 171 (step S 78 ).

Then, the envelope computing section 247 c reads all the frequency-domain separated signal Y p (f, τ) and Y p (g, τ) from the temporary memory 171 , calculates their envelopes

v p f (τ)=|Y p ( f ,τ)|

v p g (τ)=|Y p ( g ,τ)|

and stores them in the temporary memory 171 (step S 79 ).

Then, the permutation recomputing section 247 d computes the maximum sum of correlations “cor” in the vicinity less than or equal to the difference δ between the frequencies

[ Formula ⁢ ⁢ 28 ] R f = max Π ⁢ ∑  g - f  ≤ δ ⁢ ∑ k = 1 N ⁢ cor ⁡ ( v Π ⁡ ( k ) f , v Π ′ ⁡ ( k ) g )

and stores it in the temporary memory (step S 80 ). Here, Π′ is a predetermined permutation for frequency g. The correlation cor(Φ, Ψ) in the equation represents the correlation between two signals Φ and Ψ, defined as

cor(Φ,Ψ)=(<Φ,Ψ>−<Φ>·<Ψ>)/(σ Φ ·σ Ψ )

where <ζ> is the time average of ζ, σ Φ is the standard deviation of Φ.

The permutation recomputing section 247 d calculates a permutation that maximizes the sum of the correlations cor as

[ Formula ⁢ ⁢ 29 ] Π f ′ = arg ⁢ max Π ⁢ ∑  g - f  ≤ δ ⁢ ∑ k = 1 N ⁢ cor ⁡ ( v Π ⁡ ( k ) f , v Π ′ ⁡ ( k ) g )

and stores it in memory area 110 of the memory 100 (step S 81 ). Here, Π′ is a permutation predetermined for frequency g and argmax Π ν represents Π that maximizes ν.

Then the control section 170 stores the sum of sets F and {ζ} (where ζ=argmax f R f ) in the temporary memory 171 as a new set F (step S 82 ). Then, the re-sorting section 247 e sets f=ζ and rearranges the rows of the second separation matrix W′(f) in accordance with permutation Π f ′ to generate a third separation matrix W″(f), and stores it in memory area 111 of the memory 100 (step S 83 ).

The control section 170 then determines whether set F stored in the temporary memory 171 includes all discrete frequency elements f=0, f s /L, . . . , f s (L−1) (step S 84 ). If set F does not include all discrete frequency elements f=0, f s /L, . . . , f s (L−1)/L, the control section 170 returns to step S 78 . On the other hand, if set F includes all discrete frequency elements f=0, f s /L, . . . , f s (L−1)/L, the control section 170 ends processing at step S 58 . It should be noted that, instead of the method described above, any of other methods such as the method described in Japanese Patent Application Laid-Open No. 2004-145172 or “Reference literature” may be used to perform processing at step S 58 .

<Experimental Results>

Results of experiments on sound source separation according to the first and second embodiments will be given below.

[Results of First Experiment]

A first experiment is conducted using randomly arranged sensors. The experimental conditions are as shown in FIG. 14A . Four omnidirectional microphones arranged randomly were used. However, all information about the arrangement of the sensors provided to the signal separating apparatus was the maximum distance between the microphones, which was 4 cm. Three sound sources were used: English speeches are emitted through loudspeakers for three seconds. FIG. 14B shows the results in terms of SIR (signal-to-interference ratio). The larger the SIR, the better the separation performance. The results of experiments using four methods for solving the permutation problem were compared. Env indicates the result of a method using only information about the envelope |Y p (f, τ)| of separated signals, Basis indicates the result of a method using clustering of normalized basis vectors A p ″(f) (the method according to the first embodiment), Basis+Env indicates the result of a method using the combination of these two items of information to solve the problem more accurately (the method according to the second embodiment), and Optimal indicates the result of a method using an optimum permutation obtained with the knowledge of source signals s k and impulse responses h qk (r).

Comparison of the results shows that the method using only Env provides varying separation performances whereas the method using Basis according to the first embodiment provides a sufficiently good separation performance. The results obtained using the combination of Basis and Env according to the second embodiment is almost as good as that of Optimal. Thus, a high performance of blind signal separation in the frequency domain was able to be achieved according to the present invention, even when the sensors were randomly arranged.

[Results of Second Experiment]

A second experiment is conducted using orderly arranged sensors. FIG. 15A shows the experimental conditions. Three omnidirectional microphones are linearly spaced 4 cm apart. As in the first experiment, three sound sources were used: English speeches were emitted through loudspeakers for three seconds. FIG. 15B shows the results. In this experiment, comparison was made among the results obtained using six methods, including the conventional-art method described earlier in which estimates of signal source positions are clustered. DOA represents the result of a method in which the permutation problem was solved by using only estimations of DOA (direction-of-arrival) and DOA+Env represents the result obtained by using combination of estimates of DOA and information about the envelope of separated signals.

Comparison of the results of the method using DOA and the method using DOA+Env, which are conventional-art methods, with the results of the methods using Basis and Basis+Env of the present invention shows that the present invention generally provides improved performances in the orderly sensor arrangement to which the conventional approaches can be applied. It should be noted that computational cost was approximately equivalent to that in the prior-art methods.

<Features of the First and Second Embodiments>

Features of the first and second embodiments described above can be summarized as follows.

(1) Because precise information about the positions of sensors is not needed but only information about the upper limit of the distance between one reference sensor and another sensor, random arrangement of sensors can be used and positional calibration is not required; and (2) because all information obtained from basis vectors is used to perform clustering, the permutation problem can be solved more accurately, thus improving the signal separation performance.

The present invention is not limited to the embodiments described above. For example, while the Moore-Penrose generalized inverse matrix is used in the embodiments as the generalized matrix, any other generalized matrix may be used.

The first normalizing section 142 aa of the frequency normalizing section 142 a normalizes the argument of each element A qp (f) of a basis vector A p (f) on the basis of a particular element A Qp (f) of the basis vector A p (f) according to Equation (15) in the first embodiment. However, the first normalizing section 142 aa may normalize the argument of each element A qp (f) of a basis vector A p (f) on the basis of a particular element A Qp (f) of the basis vector A p (f) in accordance with the following equations:

[Formula 30]

A qp ′″( f )=| A qp ( f )|exp{ j ·(arg[ A qp ( f )· A QP *( f )])}  (27-1)

A qp ′″( f )=| A qp ( f )|exp{ j ·(arg[ A qp ( f )]−arg[ A Qp ( f )])}  (27-2)

A qp ′″( f )=| A qp ( f )|exp{ j ·Ψ(arg[ A qp ( f )/ A Qp ( f )])}  (27-3)

Here, “.*” is a complex conjugate and “Ψ{·}” is a function, preferably a monotonically increasing function, from the viewpoint of improving the precision of clustering.

The frequency normalizing section 142 a may use the following equations

[Formula  31] A qp ′ ⁡ ( f ) = ρ · arg ⁡ [ A qp ⁡ ( f ) / A Qp ⁡ ( f ) ] 4 ⁢ fc - 1 ⁢ d ( 28 ⁢ - ⁢ 1 ) A qp ′ ⁡ ( f ) = ρ · arg ⁡ [ A qp ⁡ ( f ) · A Qp * ( f ) ] 4 ⁢ fc - 1 ⁢ d ( 28 ⁢ - ⁢ 2 ) A qp ′ ⁡ ( f ) = ρ · arg ⁡ [ A qp ⁡ ( f ) ] - arg ⁡ [ A Qp ⁡ ( f ) ] 4 ⁢ fc - 1 ⁢ d ( 28 ⁢ - ⁢ 3 ) A qp ′ ⁡ ( f ) = ρ · Ψ ⁡ ( arg ⁡ [ A qp ⁡ ( f ) / A Qp ⁡ ( f ) ] ) 4 ⁢ fc - 1 ⁢ d ( 28 ⁢ - ⁢ 4 )

instead of Equation (14) to perform frequency normalization. Here, ρ is a constant (for example ρ=1).

While the norm normalizing section 142 b in the above-described embodiments performs normalization so that the norm becomes equal to 1, it may perform normalization so that the norm becomes equal to a predetermined number other than 1. Furthermore, the norm normalizing section 142 b may be not provided and therefore norm normalization may be omitted. In that case, the clustering section 143 performs clustering of frequency-normalized vectors A p ′(f). However, the norms of frequency-normalized vectors A p ′(f) are not equal. Accordingly, the clustering reference in this case is whether vectors are similar to each other only in direction, rather than both in direction and norm. This means evaluation using the degree of similarity. One example of the measure of similarity may be cosine distance

cos θ=| A p ′ H ( f )·η k |/(∥ A p ′( f )∥·∥η k ∥)

where θ is the angle between a frequency-normalized vector A p ′(f) and the vector of the centroid η k . If cosine distances are used, the clustering section 143 generates a cluster that minimizes the total sum of the cosine distances

[Formula 32]

U i =Σ A p ′(f)εC i |A p ′ H ( f )·η i |/(∥ A p ′( f )∥·∥η i ∥)

Here, the centroid η k is the average among the members of each cluster.

In the second embodiment, the reliability of a permutation for each frequency is evaluated and, for a frequency for which the reliability of the permutation is evaluated as low, the envelope of separated signals is used to calculate a new permutation. However, a permutation for all frequencies may be generated by using the envelope of separated signals, the center vectors of clusters, and normalized basis vectors.

Furthermore, the envelope of separated signals are first used to compute a permutation, the reliability of the permutation is evaluated for each individual frequency, and then the method of the first embodiment is applied to a frequency evaluated as having a low reliability permutation to calculate a new permutation for the frequency.

While the second separation matrix W′(f) is used to compute the envelope of separated signals in the second embodiment, the first separation matrix W(f) or a matrix resulting from rearrangement of the rows of the first matrix W(f) may be used to compute the envelope.

The same value of parameter d may be used for all sensors q or different values may be set for different sensors q. For example, the distance between the reference sensor and each sensor q may be set as the value of parameter d for the sensor q.

Third Embodiment (Example of the Second Aspect of the Present Invention)

The third embodiment of the present invention will be described below.

The third embodiment uses the principles described above to extract a target signal from mixed signals in which signals originated from multiple sources are mixed, without having information about the direction of the target signal.

<Configuration>

Like the signal separating apparatus in the first embodiment, a signal separating apparatus of the present embodiment is configured by loading a signal separating program into a computer of well-known von Neumann-type. FIG. 16 is a block diagram illustrating a configuration of a signal separating apparatus 1001 according to the third embodiment.

As shown in FIG. 16 , the signal separating apparatus 1001 has a memory 1100 including memory areas 1101 - 1114 , a frequency domain transforming section 1120 , a signal separating section 1130 , a target signal selecting section 1140 , a time-frequency masking section 1150 (which is equivalent to the “separated signal generating section”), a time domain transforming section 1160 , a control section 1170 , and a temporary memory 1180 . The memory 1100 and the temporary memory 1180 may be, but not limited to, at least one of a register 10 ac , a data area 10 fb of an auxiliary storage device 10 f , and a data area 10 db of a RAM 10 d ( FIG. 2 ). The frequency domain transforming section 1120 , the signal separating section 1130 , the target signal selecting section 1140 , the time frequency masking section 1150 , the time domain transforming section 1160 , the control section 1170 , and the temporary memory 1180 are configured by an OS program and a signal separating program read into a CPU 10 a ( FIG. 2 ), for example.

FIG. 17A is a block diagram illustrating a detailed configuration of the target signal selecting section 1140 shown in FIG. 16 . FIG. 17B is a block diagram illustrating a detailed configuration of the basis vector clustering section 1142 in FIG. 17A .

As shown in FIGS. 17A and 17B , the target signal selecting section 1140 includes an inverse matrix computing section 1141 (which is equivalent to the “complex vector generating section”), a basis vector clustering section 1142 , and a selecting section 1143 . The basis vector clustering section 1142 includes a frequency normalizing section 1142 a (which constitutes the “normalizing section”), a norm normalizing section 1142 b (which constitutes the “normalizing section”), a clustering section 1142 c , and a variance determining section 1142 d . The frequency normalizing section 1142 a includes a first normalizing section 1142 aa and a second normalizing section 1142 ab.

FIG. 18A is a block diagram illustrating a detailed configuration of the time-frequency masking section 1150 shown in FIG. 16 . FIG. 18B is a block diagram showing a detailed configuration of the mask generating section 1151 shown in FIG. 18A .

As shown in FIGS. 18A and 18B , the time-frequency masking section 1150 includes a mask generating section 1151 and a masking section 1152 . The mask generating section 1151 includes a whitening matrix generating section 1151 a , a whitening section 1151 b , an angle computing section 1151 c , and a function operation section 1151 d.

The solid arrows in FIGS. 16 to 18 represents actual data flows and the dashed arrows represent theoretical information flows. Flows of data inputted to and outputted from the control section 1170 and the temporary memory 1180 are not depicted. The signal separating apparatus 1001 performs processes under the control of the control section 1170 . Unless otherwise stated, the control section 1170 performs processing while reading and writing required data in the temporary memory 1180 .

<Processing>

Processing performed in the signal separating apparatus 1001 according to the third embodiment will be described below.

The assumption is that N signal sources k(kε{1, 2, . . . , N}) exist in a space and their signals s k (t) (where “t” is sampling time) are mixed and are observed at M sensors q (qε{1, 2, . . . , M}) as mixed signals x q . In the third embodiment, a target signal originating from any of the signal sources is extracted only from mixed signals x 1 (t), . . . , x M (t) and other interfering signals are suppressed to obtain a signal y(t). The number N of signal sources may be greater or less than or equal to the number M of sensors. Information about the number N of signal sources does not need to be obtained beforehand. The processing may be performed in a situation where signal sources cannot be counted.

[Outline of Processing]

FIG. 19 is a flowchart outlining a whole signal separating process according to the third embodiment. The outline of the signal separating process in the third embodiment will be described with reference to FIG. 19 .

First, mixed signals x q (t)(qε{1, . . . , M}) in the time domain observed by M sensors are stored in memory area 1101 in the memory 1100 during preprocessing. Once the signal separation is started, the frequency domain transforming section 1120 reads the time-domain mixed signals x q (t) from memory area 1101 of the memory 1100 . The frequency domain transforming section 1120 then transforms them into the frequency-domain mixed signals X q (f, τ) by using such as a short-time Fourier transformation, and stores the frequency-domain mixed signals X q (f, τ) in memory area 1102 of the memory 1100 (step S 101 ).

Then, the signal separating section 1130 reads the frequency-domain mixed signals X q (f, τ) from memory area 1102 of the memory 1100 . The signal separating section 1130 in this example applies independent component analysis (ICA) to a mixed-signal vector X(f, τ)=[X 1 (f, τ), . . . , X M (f, τ)] T consisting of the read mixed signals X q (f, τ) to calculate, for each individual frequency f, a separation matrix W(f)=[W 1 (f), . . . , W M (f)] H of M rows and M columns (where “* H ” is a complex conjugate transposed matrix of a matrix *) and a separated signal vector

Y ( f ,τ)= W ( f )· X ( f ,τ)  (30)

(step S 102 ). The calculated separation matrix W(f) is stored in memory area 1103 of the memory 1100 . The separated signals Y p (f, τ) (pε{1, . . . , M}) constituting the separated signal vector Y(f, τ)=[Y 1 (f, τ), . . . , Y M (f, τ)] T are stored in memory area 1107 . The processing at step S 102 will be detailed later.

Then, the target signal selecting section 1140 reads the separation matrix W(f) from memory area 1103 of the memory 1100 , normalizes basis vectors which are columns of the generalized inverse matrix of the separation matrix W(f), and clusters the normalized basis vectors. The target signal selecting section 1140 selects, for each frequency f, selection signals Y I(f) (f, τ) including the target signal and basis vectors A I(f) (f) corresponding to them from the separated signals in memory area 1107 of the memory 1100 on the basis of the variances of the clusters and stores them in memory area 1111 of the memory 1100 (step S 103 ). In the third embodiment, a signal from a source which is near a sensor and therefore its power observed at the sensor is dominating over signals from the other sources and is useful as information is selected as the target signal. The processing at step S 103 will be detailed later.

Then, the time-frequency masking section 1150 reads the frequency-domain mixed signals X q (f, τ) from memory area 1102 of the memory 1100 , reads the basis vectors A I(f) (f) corresponding to the selection signals Y I(f) (f, τ) from memory area 1104 , uses them to generate a time-frequency mask M(f, τ), and stores it in memory area 1112 (step S 104 ). The processing at step S 104 (processing by the time-frequency masking section 1150 ) will be detailed later.

Then, time-frequency masking section 1150 reads the selection signals Y I(f) (f, τ) selected by the target signal selecting section 1140 from memory area 1107 of the memory 1100 and the time-frequency mask M(f, τ) from memory area 1112 . The time-frequency masking section 1150 then applies the time frequency mask M(f, τ) to the selection signals Y I(f) (f, τ) to further suppress interfering signal components remaining in the selection signals Y I(f) (f, τ) to generate masked selection signals Y I(f) ′(f, τ), and stores them in memory area 1113 of the memory 1100 (step S 105 ). The processing at step S 105 (processing by time-frequency masking section 1150 ) will be detailed later.

Finally, the time domain transforming section 1160 reads the selected separated signals Y I(f) ′(f, τ) in the frequency domain from memory area 1113 of the memory 1100 , applies a transformation such as a short-time inverse Fourier transformation to them to generate separated signals y(t) in the time domain, and stores them in memory area 1114 of the memory 1100 (step S 106 ).

[Details of Processing at Step S 102 (Processing by the Signal Separating Section 1130 )]

As mentioned above, the signal separating section 1130 in this example uses independent component analysis (ICA) to compute separation matrices W(f)=[W 1 (f), . . . , W M (f)] H consisting of M rows and M columns and separated signal vectors Y(f, τ)=[Y 1 (f, τ), . . . , Y M (f, τ)] T from the mixed-signal vectors X(f, τ)=[X 1 (f, τ), . . . , X M (f, τ)]T (step S 102 ). Independent component analysis (ICA) is a method for computing a separation matrix W(f) such that the elements of a separated signal vector Y(f, τ)=[Y 1 (f, τ), . . . , Y M (f, τ)] T are then independent of one another. Various algorithms have been proposed, including the one described in Non-patent literature 4. Independent component analysis (ICA) can separate and extract more advantageously target signals of the third embodiment which are more powerful and more non-Gaussian than interfering signals, which are less powerful and more Gaussian.

[Details of Processing at Step S 103 (Processing by the Target Signal Selecting Section 1140 )]

Independent component analysis (ICA) exploits independence of signals to separate the signals. Therefore the separated signals Y p (f, τ) have ambiguity of the order. This is because the independence is retained even if the order is changed. Therefore, a separated signal corresponding to a target signal must be selected at each frequency. The target signal selecting section 1140 performs this selection through the following process.

FIG. 20 is a flowchart illustrating details of processing by the target signal selecting section 1140 in the third embodiment. With reference to FIG. 20 , processing by the target signal selecting section 1140 will be detailed below.

First, the inverse matrix computing section 1141 reads, for each frequency, a separation matrix W(f) consisting of M rows and M columns from memory area 1103 of the memory 1100 and computes its inverse matrix

W ( f ) −1 =[A 1 ( f ), . . . , A M ( f )](where the rows are A p ( f )=[ A 1p ( f ), . . . , A Mp ( f )] T )  (31)

Here, the both sides of Equation (30) are multiplied by Equation (31) to obtain the decompositions of the frequency-domain mixed signals X(f, τ) as

[ Formula ⁢ ⁢ 33 ] X ⁡ ( f , τ ) = ∑ p = 1 M ⁢ A p ⁡ ( f ) ⁢ Y p ⁡ ( f , τ ) ( 32 )

Here, A p (f) denotes basis vectors, each of which corresponds to a separated signal Y p (f, τ) at each frequency. The basis vectors A p (f) calculated according to Equation (31) are stored in memory area 1104 of the memory 1100 (step S 111 ).

Then, the basis vector clustering section 1142 normalizes all basis vectors A p (f) (p=1, . . . , M and f=0, F s /L, . . . , f s (L−L)/L). The normalization is performed so that the normalized basis vectors A p (f) form clusters that are dependent only on the positions of multiple signal sources when the convolutive mixture of signals originated from the multiple sources are approximated as a given model (for example a near-field model). In this example, frequency normalization and norm normalization similar to those used in the first embodiment are performed.

The frequency normalization is performed by the frequency normalizing section 1142 a of the basis vector clustering section 1142 ( FIG. 17B ). In particular, the frequency normalizing section 1142 a reads basis vectors A p (f) (p=1, . . . , M and f=0, f s /L, . . . , F s (L−L)/L) from memory area 1104 of the memory 1100 and normalizes them to frequency-normalized vectors A p ′(f) that are independent of frequency and stores them in memory area 1105 of the memory 1100 (step S 112 ). The normalization is performed for each element A qp (f) of the basis vector A p (f) (The normalization will be detailed later). The norm normalization is performed by the norm normalizing section 1142 b of the basis vector clustering section 1142 ( FIG. 17B ). In particular, the norm normalizing section 1142 b reads frequency-normalized vectors A p ′(f) from memory area 1105 of the memory 1100 , normalizes them to normalized basis vectors A p ″(f) whose norm has a predetermined value (1 in this example), and stores them in memory area 1106 of the memory 1100 (step S 113 ). The normalization is performed for each frequency-normalized vector A p ′(f) (The normalization will be detailed later).

After the completion of the normalization of the basis vectors, the clustering section 1142 c ( FIG. 17B ) identifies M clusters C i (iε{1, . . . , M}) formed by the normalized basis vectors A p ″(f). In this example, the clustering section 1142 c reads the normalized basis vectors A p ″(f) from memory area 1106 of the memory 1100 , clusters them into M clusters C i (i=1, . . . , M), and stores information identifying each of the clusters C i (for example information indicating normalized basis vectors A p ″(f) that belongs to the cluster) and the centroid (center vector) of the cluster C i in memory areas 1109 and 1110 , respectively, of the memory 1100 (step S 114 ). The clustering is performed so that the total sum U of sums of squares U i of the elements (normalized basis vectors A v ″ (f)) of each cluster C i and the centroid η i of the cluster C i

[ Formula ⁢ ⁢ 34 ] U = ∑ i = 1 M ⁢ U i U i = ∑ A v ″ ⁡ ( f ) ∈ C i ⁢  A v ″ ⁡ ( f ) - η i  2 ( 33 )

is minimized. The minimization can be effectively performed by using the k-means clustering described in Non-patent literature 6, for example. The centroid η i of a cluster C i can be calculated as

[Formula  35] η i = ∑ A v ″ ⁡ ( f ) ∈ C i ⁢ A v ″ ⁡ ( f ) /  C i   ∑ A v ″ ⁡ ( f ) ∈ C i ⁢ A v ″ ⁡ ( f ) /  C i   ( 34 )

where |C i | is the number of elements (normalized basis vectors A v ″ (f)) of a cluster C i and ∥*∥ is the norm of a vector “*”. While the square of the Euclidean distance is used as the distance, it may be its generalized distance, such as the Minkowski distance.

Once M clusters C i are obtained, the variance determining section 1142 d ( FIG. 17B ) selects a cluster that corresponds to the target signal and stores selection information I(f) indicating the selected cluster in memory area 1111 of the memory 1100 (step S 115 ). In the third embodiment, the variance U i /|C i | of clusters is used as an indicator to select separated signals including the target signal. That is, the normalization of basis vectors in the third embodiment is performed in such a manner that, when the convolutive mixture of signals originated from multiple sources are approximated as a predetermined model, the normalized basis vectors are dependent only on the positions of the sources. However, there are various factors in a real environment that are not reflected in such a model. The discrepancy between a real environment and a model increases as the distance from a signal source to a sensor increases. For example, wave components reflected by objects such as walls are not taken into consideration in a near-field model and the ratio of the reflected wave components to a direct wave component increases as the distance between the signal source and a sensor increases. Therefore, a model becomes improper as the signal goes away from the sensor. Consequently, signals closer to a sensor can be normalized under conditions closer to the real environment and therefore the variance of clusters caused by a discrepancy between the real environment and the model can be reduced. In the third embodiment, a signal near a sensor is selected as the target signal. Therefore, a cluster that has a smaller variance is simply selected as the cluster corresponding to the target signal. The selection procedure (step S 1115 ) will be detailed later.

After the selection information I(f) for each frequency f is computed, a selection signal Y I(f) (f, τ) at each frequency f and its corresponding basis vector A I(f) (f) are selected. In particular, the selecting section 1143 first reads the selection information I(f) from memory area 1111 of the memory 1100 . The selecting section 1143 then reads a separated signal corresponding to the selection information I(f) from memory area 1107 as the selection signal Y I(f) (f, τ), reads its corresponding basis vector A I(f) (f) from memory area 1104 , and stores them in memory area 1111 (step S 116 ).

The normalizations at step S 112 and S 113 ( FIG. 20 ) will be detailed below.

[Details of Step S 112 (Frequency Normalization)]

FIG. 21A is a flowchart illustrating details of the frequency normalization performed at step S 112 .

First, the control section 1170 ( FIG. 16 ) assigns 1 to parameter p and stores it in the temporary memory 1180 (step S 121 ). The control section 1170 also assigns 1 to parameter q and stores it in the temporary memory 1180 (step S 122 ). Then, the frequency normalizing section 1142 a ( FIG. 17B ) reads the parameters d, c, and Q described above from memory area 1108 of the memory 1100 , reads the elements A qp (f) of the basis vector A p (f) from memory area 1104 , and reads the parameters p and q from the temporary memory 1180 . The frequency normalizing section 1142 a then performs on the elements A qp (f) of the basis vector A p (f) the following calculation

[Formula  36] A qp ′ ⁡ ( f ) =  A qp ⁡ ( f )  ⁢ exp ⁡ [ j ⁢ arg ⁡ [ A qp ⁡ ( f ) / A Qp ⁡ ( f ) ] 4 ⁢ fc - 1 ⁢ d ] ( 35 )

and stores the results A qp ′f(f) in memory area 1105 of the memory 1100 as the elements A qp ′(f) of a frequency-normalized vector A p ′(f) (step S 123 ). Here, arg[·] represents an argument, exp is Napier's number, and j is an imaginary unit. In particular, the normalization is performed according to Equations (15) and (16) given earlier.

Then, the control section 1170 determines whether the value of parameter q stored in the temporary memory 1180 satisfies q=M (step S 124 ). If not q=M, the control section 1170 sets a calculation result q+1 as a new value of parameter q, stores it in the temporary memory 1180 (step S 125 ), and then returns to step S 123 . On the other hand, if q=M, the control section 1170 further determines whether p=M (step S 126 ).

If not p=M, the control section 1170 sets a calculation result p+1 as a new value of parameter p, stores it in the temporary memory 1180 (step S 127 ), and then returns to step S 122 . On the other hand, if p=M, the control section 1170 terminates processing at step S 12 . (End of the detailed description of step S 112 (frequency normalization))

[Details of Step S 113 (Norm Normalization)]

FIG. 21B is a flowchart illustrating details of the norm normalization performed at step S 113 .

First, the control section 1170 assigns 1 to parameter p and stores it in the temporary memory 1180 (step S 131 ). Then, the norm normalizing section reads the elements A qp ′(f) of the frequency-normalized vector A p ′(f) from memory area 1105 of the memory 1100 , calculates

[ Formula ⁢ ⁢ 37 ]  A p ′ ⁡ ( f )  = ∑ q = 1 M ⁢ ( A pq ′ ⁡ ( f ) ) 2 ( 38 )

to obtain the norm ∥A p ′(f)∥ of the frequency-normalized vector A p ′(f), and stores the frequency-normalized vector A p ′(f) and its norm ∥A p ′(f)∥ in the temporary memory 1180 (step S 132 ).

Then, the norm normalizing section 1142 b reads the frequency-normalized vector A p ′(f) and its norm ∥A p ′(f)∥ from the temporary memory 1180 , calculates

A p ″( f )= A p ′( f )/∥ A p ′( f )∥  (39)

and stores the calculated normalized basis vector A p ″(f) in memory area 1106 of the memory (step S 133 ). Then, the control section 1170 determines whether the value of parameter p stored in the temporary memory 1180 satisfies p=M (step S 134 ). If not p=M, the control section 1170 sets a calculation result p+1 as a new value of parameter p, stores it in the temporary memory 1180 (step S 135 ), and then returns to step S 132 . On the other hand, if p=M, the control section 1170 terminates processing at step S 113 . The reason why the normalized basis vectors A p ″(f) form clusters has been described with respect to the first embodiment. (End of the detailed description of step S 113 (norm normalization))

The normalized basis vectors A p ″(f) thus generated are independent of frequency and are dependent only on the positions of signal sources as described in the first embodiment.

[Details of Procedure for Selecting Selection Signals (Step S 115 )]

Details of the procedure for selecting selection signals (step S 115 ) mentioned above will be illustrated below.

Cluster Selection Procedure 1

A first example selects the cluster that has the smallest variance as the cluster corresponding to a target signal. FIG. 22 is a flowchart illustrating the first example.

First, the variance determining section 1142 d ( FIG. 17B ) reads information identifying clusters C i (iε{1, . . . , M}) from memory area 1109 of the memory 1100 and also reads normalized basis vectors A p ″(f)εC i and their centroids η i from memory areas 1106 and 1110 , respectively. The variance determining section 1142 d then calculates U i for each “i” according to Equation (33), counts the elements (normalized basis vectors A v ″ (f)) that belong to C i to obtain |C i |, calculates the variance of cluster C i , U i /|C i |, and stores it in the temporary memory 1180 . Then, the variance determining section 1142 d selects the smallest one of the variances U i /|C i | stored in the temporary memory 1180 and stores information indicating the cluster corresponding to the smallest variance in the temporary memory 1180 as cluster selection information

ι=argmin i U i /|C i |  (40)

(step S 141 ). In Equation (40), argmin i * represents i that minimizes the value of “*”.

Then, the control section 1170 ( FIG. 16 ) assigns 0 to parameter f and stores it in the temporary memory 1180 (step S 142 ).

Then, the variance determining section 1142 d reads the cluster selection information ι from the temporary memory 1180 and reads the centroid η ι that corresponds to the cluster selection information t from memory area 1110 of the memory 1100 . The variance determining section 1142 d also reads the normalized basis vectors A p ″(f) {pε{1, . . . , M}} from memory area 1106 of the memory 1100 . The variance determining section 1142 d then calculates, for each frequency f, selection information

I ( f )=argmin p ∥A p ″( f )−η ι ∥ 2   (41)

and stores it in memory area 1111 (step S 143 ).

Then, the control section 11170 reads parameter f from the temporary memory 1180 and determines whether f=(L−1)·f s /L (step S 144 ). If not f=(L−1)·f s /L, the control section 1170 adds f s /L to the value of parameter f, stores the result in the temporary memory 1180 as a new value of parameter f(step S 145 ), and then returns to step S 143 . On the other hand, if f=(L−1)·f/L, the control section 1170 terminates step S 115 .

Cluster Selection Procedure 2

A second example selects clusters that have variances smaller than a predetermined threshold value as the clusters corresponding to a target signal. The threshold value is for example an empirically determined value or a value based on experimental results and is stored in the memory 1100 beforehand.

The variance determining section 1142 d sorts the variances U i /|C i | of clusters in ascending or descending order by using any of well-known sorting algorithms, instead of performing step S 141 ( FIG. 22 ). The variance determining section 1142 d then reads the threshold value stored in the memory 1100 , selects clusters that have variances U i /|C i | smaller than the threshold value, and stores the set of suffixes i that correspond to the selected clusters in the temporary memory 1180 as cluster selection information ι. The rest of the procedure is the same as cluster selection procedure 1 .

Cluster Selection Procedure 3

A third example selects not only the cluster that has the smallest variance but also a predetermined number of clusters in ascending order of variance U i |C i | (for example, three clusters in ascending order of variance) as clusters corresponding to a target cluster.

The variance determining section 1142 d sorts the variances U i /|C i | of clusters in ascending or descending order using any of well-known sorting algorithms, instead of performing processing at step S 141 ( FIG. 22 ). The variance determining section 1142 d then selects a predetermined number of clusters in ascending order of variance U i /|C i |. Then, the variance determining section 1142 d stores the set of suffixes i corresponding to the selected clusters in the temporary memory 1180 as cluster selection information ι. The rest of the procedure is the same as cluster selection procedure 1 .

In stead of cluster selection procedure 1 , a procedure which selects any of the clusters that have the second smallest variance or larger may be used, or a combination of parts of the cluster selection procedures described above may be used. (End of the description of Step S 115 and of details of step S 103 (processing by the target signal selecting section 1140 )

[Details of Processing by the Time-Frequency Masking Section 1150 (Steps S 104 and S 105 )]

Processing by the time-frequency masking section 1150 will be described below. As mentioned earlier, the time-frequency masking section 1150 suppresses interfering signal components remaining in selection signals Y I(f) (f, τ) selected by the target signal selecting section 1140 . The reason why interfering signals remain in the selection signals Y I(f) (f, τ) will be described first.

Focusing only on selection signals, equation (30) given above can be rewritten as

Y I(f) ( f ,τ)= W I(f) H ( f )· X ( f ,τ)  (42)

If Equation (4) is substituted in Equation (42) and frequency f is omitted, the equation can be rewritten as.

[Formula  38] Y I ⁡ ( τ ) = W I H · H I · S I ⁡ ( τ ) + ∑ k = I , ⁢ … ⁢ , I - 1 , I + 1 , ⁢ … ⁢ , N ⁢ W I H · H k · S k ⁡ ( τ ) ( 43 )

If N≦M, W 1 that satisfies W 1 H ·H k =0, ∀ kε{1, . . . , I−1, I+1, . . . , N} can be set by using independent component analysis (ICA). Then, the second term in Equation (43) will be 0. However, if the number N of signal sources is greater than the number M of sensors, which is a more common situation, there is κ ⊂ {1, . . . , I−1, I+1, . . . , N} that results in W 1 H ·H k ≠0, ∀ kεκ. In this case, selection signals Y 1 (f) include unnecessary residual components (residual components of interfering signals)

[ Formula ⁢ ⁢ 39 ] ∑ k ∈ κ N ⁢ W I H · H k · S k ⁡ ( τ )

(hereinafter f is not omitted).

The purpose of using the time-frequency masking section 1150 is to suppress such unnecessary residual components included in selection signals Y 1 (f, τ), thereby generating masked selection signals Y 1 ′(f, τ) including less residual interfering signal components. For this purpose, the mask generating section 1151 ( FIG. 18 ) of the time-frequency masking section 1150 generates a time-frequency mask 0≦M(f, τ)≦1 that takes on a smaller value for a time-frequency slot containing more residual interfering signal components and a greater value for a time-frequency slot containing less residual interfering signal components. Then, the masking section 1152 performs masking in accordance with

Y I(f) ′( f ,τ)= M ( f ,τ)· Y I(f) ( f ,τ)  (44)

and outputs masked selection signals Y I(f) ′(f, τ). The mask generation will be detailed below.

[Details of Step S 104 (Processing by Mask Generating Section 1151 )]

FIG. 23 is a flowchart illustrating details of step S 104 in FIG. 19 . With reference to the flowchart, step S 104 (processing by the mask generating section 1151 ) will be detailed below.

The mask generating section 1151 in this example obtains the angle θ I(f) (f, τ) between a mixed-signal vector X(f, τ) and a basis vector A I(f) (f) corresponding to a selection signal in a space in which the frequency-domain mixed-signal vector X(f, τ) is whitened (a whitening space), and generates a time-frequency mask based on the angle θ I(f) (f, τ). Whitening transforms a mixed-signal vector X(f, τ) into a linear form so that its covariance matrix becomes equal to an identity matrix.

For that purpose, first the whitening matrix generating section 1151 a uses frequency-domain mixed signals X q (f, τ) to generate a whitening matrix V(f) which transfers a mixed-signal vector X(f, τ) into a whitening space (step S 151 ). In this example, the whitening matrix generating section 1151 a reads the mixed signals X q (f, τ) from memory area 1102 of the memory 1100 , computes V (f) =R(f) −1/2 , where R(f)=<X(f, τ)·X(f, τ) H >τ, as a whitening matrix V(f), and stores it in memory area 1112 . Here, <*> τ represents the time-averaged vector of a vector “*”, “* H ” represents the complex conjugate transposed matrix of the vector “*”, R −1/2 represents a matrix that satisfies R −1/2 ·R·(R −1/2 ) H =I (where I is the identity matrix). A typical method for calculating the whitening matrix V(f) is to decompose R(f) into eigenvalues as R(f)=E(f)·D(f)·E(f) H (where E(f) is an unitary matrix and D(f) is a diagonal matrix) and calculate V(f)=D(f) −1/2 ·E(f) H . Here, D(f) −1/2 is equivalent to a diagonal matrix obtained by raising each element of the diagonal matrix D(f) to the (−½)-th power and therefore can be calculated by raising each element to the (−½)-th power.

Then, the whitening section 1151 b uses the whitening matrix V(f) to map the mixed-signal vector X(f, τ) to the whitening space to obtain a whitened mixed-signal vector Z(f, τ) and map the basis vector A I(f) (f) to the whitening space to obtain a whitened basis vector B I(f) (f) (step S 152 ). In this example, the whitening section 1151 b first reads mixed signals X q (f, τ) from memory area 1102 of the memory 1100 , the basis vectors A I(f) (f) corresponding to selection signals Y I(f) (f, τ) from memory area 1111 , and the whitening matrix V(f) from memory area 1112 . The whitening section 1151 b then calculates a whitened mixed-signal vector Z(f, τ) using the operation Z(f, τ)=V(f)·X(f, τ), calculate a whitened basis vector B I(f) (f) using the operation B I(f) (f)=V(f)·A I(f) (f), and then stores them in memory area 1112 of the memory 1100 .

Then, the angle computing section 1151 c computes the angle θ I(f) (f, τ) between the whitened mixed-signal vector Z(f, τ) and the whitened basis vector B I(f) (f) for each time-frequency (step S 153 ). In this example, the angle computing section 1151 c first reads the whitened mixed-signal vector Z(f, τ) and the whitened basis vector B I(f) (f) from memory area 1112 of the memory 1100 . The angle computing section 1151 c then calculates the angle θ I(f) (f, τ) in each time-frequency slot as

θ I(f) ( f ,τ)=cos −1 (| B I(f) H ( f )· Z ( f ,τ)|/∥ B I(f) ( f )∥·∥ Z ( f ,τ)  (45)

and stores it in memory area 1112 . In Equation (45), |*| represents the absolute value of a vector “*” and ∥*∥ represents the norm of the vector “*”.

Then, the function operation section 1151 d generates a time-frequency mask M(f, τ), which is a function including the angle θ I(f) (f, τ) as an element (step S 154 ). In this example, the function operation section 1151 d first reads real-number parameters θ T and g from memory area 1108 of the memory 1100 and the angle θ I(f) (f, τ) from memory area 1112 . The function operation section 1151 d then calculates a logistic function

M ⁡ ( θ ⁡ ( f , τ ) ) = 1 1 + ⅇ g · ( θ ⁡ ( f , τ ) - θ T ) ( 46 )

as the time-frequency mask M(f, τ). The real-number parameters θ T and g are parameters that specify the turning point and gradient, respectively, of the time-frequency mask M(f, τ), and are stored in memory area 1108 during preprocessing. FIG. 24A shows an exemplary time-frequency mask M(f, τ) calculated using the two real-number parameters θ T , and g according to Equation (46). As shown, the smaller the real-number parameter θ T , the narrower the area where the time-frequency mask M(f, τ) takes on a large value (1 in this example). This appears in the tendency that, as the value of the real-number parameter θ T decreases, the quantity of interfering signal components remaining in the masked selection signal Y I(f) ′(f, τ) decreases but at the same time the masked selection signal Y I(f) ′(f, τ) becomes unnatural. For example, if the target signal is a speech signal, musical noise increases as the value of the real-number parameter θ T decreases. Furthermore, the waveform of the time-frequency mask M(f, τ) (transition from a large value (1 in this example) to a small value (0 in this example)) steepens with increasing value of the real-number parameter g. To minimize interfering signal components remaining in the masked selection signal Y I(f) ′(f, τ) while keeping the masked selection signal Y I(f) ′(f, τ) natural, it is desirable to smoothen the waveform of the time-frequency mask M(f, τ) by minimizing the value of the real-number parameter g.

Values of the real-number parameters θ T and g may be set for each frequency. An additional real-number parameter α may be introduced and the logistic function

M ⁡ ( θ ⁡ ( f , τ ) ) = α 1 + ⅇ g · ( θ ⁡ ( f , τ ) - θ T ) ( 47 )

may be used as the time-frequency mask M(f, τ). Any other function may be used as the time-frequency mask M(f, τ) that takes on a larger value in a region where the angle θ I(f) (f, τ) is close to 0 and takes on a smaller value in a region where the angle θ I(f) (f, τ) is large, that is, 0≦M(θ(f, τ))≦1. (End of the detailed description of step S 104 (processing by the mask generating section 1151 )

[Details of Step S 105 (Processing by the Masking Section 1152 )]

The masking section 1152 reads the selection signal Y I(f) ′(f, τ) from memory area 1111 of the memory 1100 and the time-frequency mask M(f, τ) from memory area 1112 . The masking section 1152 then calculates a masked selection signal Y I(f) ′(f, τ) as

Y I(f) ′( f ,τ)= M ( f ,τ)· Y I(f) ( f ,τ)  (48)

and stores it in memory area 1113 of the memory 1100 . (End of the detailed description of step S 105 (processing by the masking section 1152 ))

[Effects of the Time-Frequency Masking]

Effects of the time-frequency mask M(f, τ) described above will be described next.

If the sparseness of signal sources is so high that the signal sources S k (f, τ) is likely to approach 0, Equation (4) can be approximated as

[Formula 40]

X ( f ,τ)≈ H k ( f )· S k ( f ,τ), kε{ 1, . . . , N}   (49)

where k is the suffix associated with each signal source and is determined by each time-frequency position (f, τ). Accordingly, in a time-frequency position (f, τ) where only or practically only the target signal is active, the whitened mixed vector Z(f, τ) can be approximated as

[Formula 41]

Z ( f ,τ)≈ V ( f )· H I(f) ( f )· S I(f) ( f ,τ)≈ V ( f )· A I(f) ( f )· Y I(f) ( f ,τ)

where Y I(f) (f, τ) is a scalar. As mentioned above, the whitened basis vector B I(f) (f) is

B I(f) ( f )= V ( f )· A I(f) ( f )  (50)

It can be seen from the foregoing that the angle θ I(f) (f, τ) between a whitened mixed-signal vector Z(f, τ) and a whitened basis vector B I(f) (f) approaches 0 at a time-frequency position (f, τ) where only or practically only the target signal is active. As stated above, the time-frequency mask M(f, τ) takes on a larger value in a region where the angle θ I(f) (f, τ) is closer to 0. Therefore, the time-frequency mask M(f, τ) extracts a selection signal Y I(f) (f, τ) at a time-frequency position (f, τ) where only or practically only the target signal is active as a masked selection signal Y I(f) ′(f, τ) (see Equation (48)).

On the other hand, if I(f)=1, the whitened mixed-signal vector Z(f, τ) in a time-frequency position (f, τ) where the target signal is almost inactive can be approximated as

[ Formula ⁢ ⁢ 42 ] Z ⁡ ( f , τ ) ≈ ∑ k = 2 N ⁢ V ⁡ ( f ) · H k ⁡ ( f ) · S k ⁡ ( f , τ ) ( 51 )

Here, if the number N of signal sources is equal to or less than the number M of sensors, vectors V(f)·H 1 (f), . . . , V(f)·H k (f) in a whitening space are orthogonal to each other. S k (f, τ) in Equation (51) is a scalar value. Thus, it can be seen that the angle θ I(f) (f, τ) between the whitened mixed-signal vector Z(f, τ) and the whitened basis vector B I(f) (f) increases. If N>M, the whitened basis vector B I (f)(I(f)=1) tends to form a large angle with vectors V(f)·H 2 (f), . . . , V(f)·H k (f) other than the target signal. It can be seen from the foregoing that the angle θ I(f) (f, τ) takes on a large value at a time-frequency position (f, τ) where the target signal is almost inactive. Because the time-frequency mask M(f, τ) takes on a small value in a region where the angle θ I(f) (f, τ) is far from 0, the time-frequency mask M(f, τ) excludes a selection signal Y I(f) (f, τ) at a time-frequency position (f, τ) where the target signal is almost inactive from a masked selection signal Y I(f) ′(f, τ) (see Equation (28)).

It can be seen from the foregoing that the time-frequency masking using the time-frequency mask M(f, τ) further suppresses interfering signal components remaining in the selection signal Y I(f) (f, τ).

The time-frequency masking is effective especially for signals having sparseness such as speech or music. Less sparse signals contain a large quantity of other interfering signal components even in a time-frequency position (f, τ) where a target signal is active, therefore the approximation by Equation (49) cannot hold and the angle θ I(f) (f, τ) will be far from 0. That is, if a signal is not sparse, vectors V(f)·H 2 (f) and V(f)·H 3 (f) corresponding to interfering signals exist together with the vector V(f)·H 1 (f) corresponding to the target signal (I(f)=1) in a time-frequency position (f, τ) as shown in FIG. 24B , for example. In this example, the whitened mixed-signal vector Z(f, τ) is

[ Formula ⁢ ⁢ 43 ] Z ⁡ ( f , τ ) ≈ ∑ k = 1 3 ⁢ V ⁡ ( f ) · H k ⁡ ( f ) · S k ⁡ ( f , τ ) ( 52 )

Therefore, the angle θ I(f) (f, τ) between the whitened mixed-signal vector Z(f, τ) and the whitened basis vector B I(f) (f) is also far from 0. This shows that a signal at a time-frequency position (f, τ) where the target signal is active can be excluded from masked selection signals Y I(f) ′(f, τ).

The time-frequency masking is also especially effective in a case where the power of a target signal is sufficiently large compared with that of interfering signals. That is, even in a situation where sparseness is low and other interfering signal components exist at a time-frequency position (f, τ) where the target signal is active, the approximation by Equation (49) is relatively likely to hold and the angle θ I(f) (f, τ) approaches 0 if the power of the target signal is sufficiently large compared with that of the interfering signals. For example, if the power of the target signal is sufficiently large compared with the power of interfering signals, the contribution of the interfering signals in Equation (52) is low and the angle θ I(f) (f, τ) between the whitened mixed-signal vector Z(f, τ) and the whitened basis vector B I(f) (f) approaches 0. This shows that the possibility that the signals at time-frequency position (f, τ) where the target signal is active will be excluded from the masked selection signals Y I(f) ′(f, τ) can be decreased. It also means that interfering signal components remaining in the masked selected signal Y I(f) ′(f, τ) can be reduced to a relatively low level. (End of detailed description of Step S 105 (processing by the masking section 1152 )

Fourth Embodiment (Example of the Second Aspect of the Invention)

The fourth embodiment of the present invention will be described below.

The fourth embodiment is a variation of the third embodiment and is the same as the third embodiment except that time-frequency masking using a time-frequency mask is not performed. The following description will focus on differences from the third embodiment and the description of the same elements as those in the third embodiment will be omitted.

<Configuration>

FIG. 25 is a block diagram showing an exemplary signal separating apparatus 1200 according to the fourth embodiment.

As shown in FIG. 25 , the signal separating apparatus 1200 of the fourth embodiment differs from the signal separating apparatus 1001 in that the memory 1100 does not include memory areas 1112 and 1113 and the time-frequency masking section 1150 is not provided.

<Processing>

Processing performed in the signal separating apparatus 1200 according to the fourth embodiment will be described below.

FIG. 26 is a flowchart illustrating processing performed in the signal separating apparatus 1200 according to the fourth embodiment. The following description focuses on differences from the third embodiment.

First, as in the third embodiment, a frequency domain transforming section 1120 reads time-domain mixed signals x q (t) from memory area 1101 of a memory 1100 . The frequency domain transforming section 1120 then transforms them into frequency-domain mixed signals X q (f, τ) using a transformation such as a short-time Fourier transformation and stores them in memory area 1102 of the memory 1100 (step S 161 ).

Then, a signal separating section 1130 reads the frequency-domain mixed signals X q (f, τ) from memory area 1102 of the memory 1100 . The signal separating section 1130 in this example applies independent component analysis (ICA) to a mixed-signal vector X(f, τ)=[X 1 (f, τ), . . . , X M (f, τ)] T consisting of the read mixed signals X q (f, τ) to calculate a separation matrix of M rows and M columns W(f)=[W 1 (f), . . . , W M (f)] H (where “* H ” is the complex conjugate transposed matrix of a matrix “*”) and a separated signal vector Y(f, τ)=W(f)·X(f, τ) for each frequency f (step S 162 ). The calculated separation matrix W(f) is stored in memory area 1103 of the memory 1100 . The separated signals Y p (f, τ)(pε{1, . . . , M}) constituting the separated signal vector Y(f, τ)=[Y 1 (f, τ), . . . , Y M (f, τ)] T are stored in memory area 1107 .

Then, a target signal selecting section 1140 reads the separation matrix W(f) from memory area 1103 of the memory 1100 , normalizes basis vectors which are rows of its generalized inverse matrix, and clusters the normalized basis vectors. The target signal selecting section 1140 then selects selection signals Y I(f) (f, τ) from the separated signals in memory area 1107 of the memory 1100 for each frequency using the variance of the clusters as the reference and stores them in memory area 1111 of the memory 1100 (step S 1163 ).

Then, a time domain transforming section 1160 reads the selected separated signals Y I(f) (f, τ) from memory area 1111 of the memory 1100 and applies a transformation such as a short-time inverse Fourier transformation to them to generate time-domain separated signals y(t), and stores them in memory area 1114 of the memory 1100 (step S 164 ).

Fifth Embodiment (Example of the Second Aspect of the Invention)

The fifth embodiment of the present invention will be described below.

The fifth embodiment is a variation of the third embodiment. The only difference from the third embodiment is the method for generating a time-frequency mask. The following description will focus on differences from the third embodiment and description of the same elements as those in the third embodiment will be omitted.

<Configuration>

FIG. 27 is a block diagram showing an exemplary signal separating apparatus 1300 according to the fifth embodiment. FIG. 28A is a block diagram showing a detailed configuration of a time-frequency masking section 1350 shown in FIG. 27 . FIG. 28B is a block diagram showing a detailed configuration of a mask generating section 1351 shown in FIG. 28A . In these drawings, the same elements as those in the third embodiments are labeled with the same reference numerals used in the drawings of the third embodiment.

As shown in FIG. 27 , the signal separating apparatus 1300 of the fifth embodiment differs from the signal separating apparatus 1001 in that the signal separating apparatus 1300 has the time-frequency masking section 1350 instead of the time-frequency masking section 1150 , and that the memory 1100 has memory areas 1308 and 1312 instead of memory areas 1108 and 1112 . As shown in FIG. 28A , the time-frequency masking section 1350 includes the mask generating section 1351 and the masking section 1152 . As shown in FIG. 28B , the mask generating section 1351 includes a frequency normalizing section 1351 a , a norm normalizing section 1351 b , a centroid extracting section 1351 c , a squared-distance computing section 1351 d , and a function generating section 1351 e . The frequency normalizing section 1351 a includes a first normalizing section 1351 aa and a second normalizing section 1351 ab . The centroid extracting section 1351 c includes a centroid selecting section 1351 ca and a norm normalizing section 1351 cd.

<Mask Generation>

The fifth embodiment differs from the third embodiment only in time-frequency mask generation (step S 104 ). The time-frequency mask generation of the fifth embodiment will be described below.

FIG. 29 is a flowchart illustrating a process for generating a time-frequency mask according to the fifth embodiment. FIG. 30A is a flowchart illustrating details of step S 171 in FIG. 29 . FIG. 30B is a flowchart illustrating details of step S 172 in FIG. 29 . Referring to these flowcharts, the time-frequency mask generating process will be described below.

First, the frequency normalizing section 1351 a of the mask generating section 1351 normalizes a mixed-signal vector X(f, τ) consisting of frequency-domain mixed signals X q (f, τ) stored in memory area 1102 of the memory 1100 to a frequency-normalized vector X′(f, τ) that is independent of frequency (frequency normalization) and stores the elements X q ′(f, τ) of the frequency-normalized vector X′(f, τ) in memory area 1312 of the memory 1100 (step S 171 ).

[Details of Frequency Normalization (Step S 171 )]

The frequency normalization (step S 171 ) will be detailed below.

First, a control section 1170 ( FIG. 27 ) assigns 1 to parameter q and stores it in a temporary memory 1180 (step S 181 ). Then, the frequency normalizing section 1351 a ( FIG. 28B ) reads parameters d, c, and Q described earlier from memory area 1308 of the memory 1100 , reads the elements X q (f, T) of the mixed-signal vector X(f, τ) corresponding to each (f, τ) from memory area 1102 , and reads the parameter q from the temporary memory 1180 . The frequency normalizing section 1351 a then calculates

[ Formula ⁢ ⁢ 44 ] X q ′ ⁡ ( f , τ ) =  X q ⁡ ( f , τ )  ⁢ exp ⁡ [ j ⁢ arg ⁡ [ X q ⁡ ( f , τ ) / X Q ⁡ ( f , τ ) ] 4 ⁢ ⁢ fc - 1 ⁢ d ] ( 53 )

and stores the result in memory area 1312 of the memory 1100 as each element of a frequency-normalized vector X′(f, τ)=[X 1 ′(f, τ), . . . , X M ′(f, τ)] T (step S 182 ). Here, arg[·] represents an argument and j represents an imaginary unit.

In particular, the first normalizing section 1351 aa of the frequency normalizing section 1351 a normalizes the argument of each element X q (f, τ) of a mixed-signal vector X(f, τ) by using one particular element X q (f, τ) of the mixed-signal vector X(f, τ) as a reference according to the following operation.

[Formula 45]

X q ′″( f ,τ)=| X q ( f ,τ)|exp{ j ·arg[ X q ( f ,τ)/ X Q ( f ,τ)]}  (54)

Then, the second normalizing section 1351 ab of the frequency normalizing section 1351 a divides the argument of each of the elements X q ′″(f, τ) normalized by the first normalizing section 1351 aa by a value 4fc −1 proportional to the frequency f, as follows.

Then, the control section 1170 determines whether the value of parameter q stored in the temporary memory 1180 satisfies q=M (step S 183 ). If not q=M, the control section 1170 sets a calculation result q+1 as a new value of the parameter q, stores it in the temporary memory 1180 (step S 184 ), and then returns to step S 182 . On the other hand, if q=M, the control section 1170 terminates processing at step S 171 and causes processing at step S 172 , described below, to be performed. (End of the detailed description of the frequency normalization (step S 171 )

Then, the norm normalizing section 1351 b of the mask generating section 1351 normalizes a frequency-normalized vector X′(f, τ) consisting of the elements X q ′(f, τ) stored in memory area 1312 of the memory 1100 to a norm-normalized vector X″(f, τ) whose norm has a predetermined value (1 in this example) (norm normalization) and stores the elements X q ″ (f, τ) in memory area 1312 (step S 172 ).

[Details of Norm Normalization (step S 172 )]

The norm normalization (step S 172 ) will be detailed below.

First, the norm normalizing section 1351 b ( FIG. 28B ) reads the frequency-normalized vectors X′(f, τ)=[X 1 ′(f, τ), . . . , X M ′(f, τ)] T each of which corresponds to (f, τ), from memory area 1312 of the memory 1100 . The norm normalizing section 1351 b then calculates their norms ∥X′(f, τ)∥ as

[ Formula ⁢ ⁢ 47 ]  X ′ ⁡ ( f , τ )  = ∑ q = 1 M ⁢ ( X q ′ ⁡ ( f , τ ) ) 2

and stores the frequency-normalized vectors X′(f, τ) and the norms ∥X′(f, τ)∥ in the temporary memory 1180 (step S 185 ).

Then, the norm normalizing section 1351 b reads the frequency-normalized vector X′(f, τ) corresponding to each (f, τ) and its norm ∥X′(f, τ)∥ from the temporary memory 1180 and calculates a norm-normalized vector X″ (f, τ) as

X″ ( f ,τ)= X ′( f ,τ)/∥ X ′( f ,τ)∥

(step S 186 ).

The calculated norm-normalized vector X″ (f, τ) is stored in memory area 1312 of the memory 1100 . With this, step S 172 ends. (End of the detailed description of the norm normalization (step S 172 ))

Then, a centroid selecting section 1351 ca of a centroid extracting section 1351 c reads cluster selection information ι from the temporary memory 1180 (see step S 141 ) and reads a centroid η ι , corresponding to the cluster selecting information t from memory area 1110 of the memory 1100 (step S 173 ). Then, the norm normalizing section 1351 cb normalizes the norm of the centroid, read by the centroid selecting section 1351 ca to a predetermined value (the value at step S 172 , which is 1 in this example). The centroid η ι after norm normalization is referred to as a norm-normalized centroid η 76 ′ (step S 174 ). The procedure for norm normalization is the same as the procedure at steps S 185 and S 186 . The norm-normalized centroid η ι ′ is stored in memory area 1312 of the memory 1100 .

Then, the squared distance computing section 1351 d reads the norm-normalized vector X″ (f, τ) and the norm-normalized centroid η ι ′ from memory area 1312 of the memory 1100 and computes the squared distance between them as

DS ( f ,τ)=∥η ι ′−X ″( f ,τ)∥ 2

(step S 175 ) and stores the squared distance DS(f, τ) in memory area 1312 .

Then, the function generating section 1351 e reads the squared distance DS(f, τ) from memory area 1312 of the memory 1100 , uses a function having the squared distance DS(f, τ) as its variable to generate a time-frequency mask M(f, τ), and stores it in memory area 1312 of the memory 1100 (step S 176 ). In particular, the function generating section 1351 e reads real-number parameters g and D T from memory area 1308 of the memory 1100 and generates a time-frequency mask M(DS(f, τ)), which is a logistic function as given below. Here, the parameter DT has been stored previously in memory area 1308 and “e” is Napier's number.

The time-frequency mask M(DS(f, τ)) thus generated is used in masking in the masking section 1152 as in the third embodiment.

[Experimental Results]

In order to demonstrate effects of the third and fourth embodiments, experiments were conducted to enhance and extract main speech emitted near microphones. In the experiments, impulse responses h qk (r) were measured under the conditions shown in FIG. 31 A. Loudspeakers were arranged in such a manner that a cocktail party situation is simulated. Furthermore, the sound levels of all loudspeakers were set to approximately equal values so that a particular loudspeaker did not output sound at a significantly higher volume level than the others. Mixing at the microphones was generated by convolving English speech sampled at 8 kHz for 6 seconds with measured impulse responses. The microphones were arranged three-dimensionally as shown in FIG. 31A . A system (apparatus) containing the signal separating apparatus was supplied with only information about the maximum distance (3.5 cm) between the reference microphone (Mic.2) and other microphones but not with further information about the layout of the microphones. In each experiment, one of the four loudspeaker positions (a 120 , b 120 , c 120 , and c 170 ) near the microphones was selected as a target sound source and the other three loudspeakers were kept silent. Six loudspeakers distant from the microphones were outputting interfering sounds at all times during the experiments. The results of the extraction were evaluated on the basis of improvements in the signal-to-interference ratio, Input SIR-Output SIR. Greater values mean better extraction of a target speech and therefore higher levels of suppression of the other interfering sounds. The two kinds of SIR are defined by

[ Formula ⁢ ⁢ 49 ] InputSIR = 10 ⁢ log 10 ⁢ 〈  ∑ r ⁢ h 11 ⁡ ( r ) · s 1 ⁡ ( t - r )  2 〉 t 〈  ∑ k ≠ 1 ⁢ ⁢ ∑ r ⁢ h 1 ⁢ k ⁡ ( r ) · s k ⁡ ( t - r )  2 〉 t ⁢ ( dB ) OutputSIR = 10 ⁢ log 10 ⁢ 〈  ∑ r ⁢ u 11 ⁡ ( r ) · s 1 ⁡ ( t - r )  2 〉 t 〈  ∑ k ≠ 1 ⁢ ⁢ ∑ r ⁢ u 1 ⁢ k ⁡ ( r ) · s k ⁡ ( t - r )  2 〉 t ⁢ ( dB ) where u 1 ⁢ k ⁡ ( r ) = ∑ q = 1 M ⁢ ⁢ ∑ τ = 0 L - 1 ⁢ ⁢ w 1 ⁢ q ⁡ ( τ ) · h qk ⁡ ( r - τ )

is an impulse response from s k (t) to y 1 (t).

Sixteen combinations, each consisting 7 speeches (1 target speech and 6 interfering speeches), were created for each target sound source position for the experiments. FIG. 31B is a table showing average improvements in SIR in the case where only ICA was used (the fourth embodiment) and in the case where both ICA and time-frequency masking were used (the third embodiment). Generally good improvements in SIR were yielded, with slight variations depending on the positions of the target sound sources. The good results were obtained at positions a 120 and b 120 because the interfering sounds came from different positions. From a two-dimensional perspective, positions c 120 and c 170 appear to be positions where it is difficult to extract the target speech because many interfering sounds come from the same direction. Actually, the results obtained at a position c 170 were excellent, however. This is because position c 170 was placed at a height different from those of interfering sounds and the system automatically uses the difference in height to extract signals with the three dimensionally arranged microphones. The table in FIG. 31B shows that the performance is improved by the time-frequency masking. Three parameters shown in FIG. 31A were used in Equation (46) that determines a time-frequency mask. By using smaller values of θ T , greater SIR improvements are achieved. However, some of sounds obtained using smaller θ T were accompanied by unnatural noise (musical noise). The experiments showed that parameters (θ t , g)=(0.333π, 20) sufficiently suppressed interfering sounds while providing natural sound.

[Variations]

The present invention is not limited to the third to fifth embodiments described above. For example, while the signal separating section 1130 computes a separation matrix W(f) consisting of M rows and M columns in the embodiments described above, it may compute a non-square separation matrix W(f) such as a matrix consisting of N rows and M columns. In that case, basis vectors are the columns of a generalized inverse matrix W + (f) (for example, a Moore-Penrose generalized matrix) of the separation matrix W(f).

While a time-frequency mask is used to further suppress interfering signal components in selection signals Y I(f) (f, τ) to generate masked selection signals Y I(f) ′(f, τ) in the third embodiment, any other method may be used to suppress interfering signal components to generate masked selection signal Y I(f) ′(f, τ). For example, if there are only two signal sources, a time-frequency mask may be generated that compares the magnitude of extracted separated signals Y 1 (f, τ) and Y 2 (f, τ), and extracts Y 1 (f, τ) as the masked selection signal Y I(f) ′(f, τ) if |Y 1 (f, τ)|>|Y 2 (f, τ|, or extracts the signal Y 2 (f, τ) as the masked selection signal Y 1 (f)′(f, τ) if |Y(f, τ)<|Y 2 (f, τ)|. Then, vectors consisting of the separated signals Y 1 (f, τ) and Y 2 (f, τ) is multiplied by the generated time-frequency mask.

While the signal separating section 1130 uses independent component analysis (ICA) to compute a separation matrix and separated signals in the third embodiment, it may use a time-frequency mask (which is a mask for each time frequency, for example a binary mask that takes on the value 1 or 0) to extract separated signals from observed signals (for example see O. Yilmaz and S. Richard, “Blind separation of speech mixtures via time-frequency masking,” IEEE Trans. an SP. vol. 52, no. 7, pp. 1830-1847, 2004) and may generate a separation matrix from the result. The first normalizing section 1142 aa of the frequency normalizing section 1142 a in the third embodiment normalizes the arguments of the components A qp (f) of a basis vector A p (f) by using one particular element A Qp (f) of that basis vector A p (f) as the reference according to Equation (15), which is a part of Equation (35). However, the first normalizing section 1142 aa may use a particular element A Qp (f) of a basis vector A p (f) as the reference to normalize the arguments of the components A qp (f) of that basis vector A p (f) according to Equations (27-1) to (27-3) described above.

Furthermore, the frequency normalizing section 1142 a may perform frequency normalization by calculating Equations (28-1) to (28-4) given above, instead of Equation (35).

While the norm normalizing section 1142 b performs normalization such that a norm has a value of 1 in the third embodiment, it may perform normalization such that a norm has a predetermined value other than 1. Furthermore, the norm normalizing section 1142 b is not provided and therefore norm normalization may be omitted. In this case, clustering is performed on the basis of the similarity in the directions of vectors as described above.

The same value of parameter d may be set for all sensors q or different values may be set for different sensors q. For example, the distance between the reference sensor and a sensor q may be set the value of parameter d at the sensor q.

Sixth Embodiment (Example of the Third Aspect of the Invention)

The sixth embodiment of the present invention will be described below.

The sixth embodiment uses the principles described above and uses information obtained from all observed signals in a simple and efficient manner to perform signal separation without needing precise positional information about sensors. In the sixth embodiment, a “mixed-signal vector” which will be described later corresponds to the “complex vector” described above.

<Configuration>

Like the signal separating apparatus of the first embodiment, a signal separating apparatus 2001 of the sixth embodiment is configured by loading a signal separating program into a computer of well-known von Neumann-type. FIG. 32 is a block diagram showing an exemplary configuration of the signal separating apparatus 2001 in the sixth embodiment. FIG. 33 is a block diagram illustrating details of a signal separating section 2120 shown in FIG. 32 . The solid arrows in FIGS. 32 and 33 represent actual data flows and the dashed arrows represent theoretical information flows. Arrows representing flows of data inputted in and outputted from a control section 2140 are omitted from FIGS. 32 and 33 .

As shown in FIGS. 32 and 33 , the signal separating apparatus 2001 includes a memory 2100 , a frequency domain transforming section 2110 (including the functions of the “complex vector generating section”), the signal separating section 2120 , a time frequency transforming section 2130 , and the control section 2140 . The signal separating section 2120 includes a frequency normalizing section 2121 (constituting the “normalizing section”), a norm normalizing section 2122 (constituting the “normalizing section”), a clustering section 2123 , and a separated signal generating section 2124 . The frequency normalizing section 2121 includes a first normalizing section 2121 a and a second normalizing section 2121 b . The control section 2140 has a temporary memory 2141 .

The memory 2100 and the temporary memory 2141 correspond to storage such as a register 10 ac , an auxiliary storage device 10 f , and a RAM 10 d . The frequency domain transforming section 2110 , the signal separating section 2120 , the time domain transforming section 2130 , and the control section 2140 are configured when an OS program and the signal separating program are read in the CPU 10 a and the CPU 10 a executes them.

<Processing>

Processing performed in the signal separating apparatus 2001 will be described below. In the following description, a situation will be dealt with in which N source signals are mixed and observed by M sensors. The assumption is that mixed signals X q (t) (q=1, . . . , M) in the time domain observed at the sensors are stored in memory area 2101 of the memory 2100 and signal transmission speed c, reference values Q and Q′ selected from natural numbers less than or equal to M (each being the suffixes indicating reference sensors selected from among the M sensors) and values of real-number d parameters are stored in memory area 2105 .

FIG. 34 is a flowchart outlining whole processing in the signal separating apparatus 2001 according to the sixth embodiment. The processing by the signal separating apparatus 2001 of the sixth embodiment will be described with reference to the flowchart.

[Overview of Processing]

First, the frequency domain transforming section 2110 reads mixed signals X q (t) in the time domain from memory area 2101 of the memory 2100 , transforms them into time-series signals of individual frequency (referred to as “frequency-domain mixed signals) X q (f, τ)(q=1, . . . , M and f=0, f s /L, . . . , f s (L−1)L, where f s is a sampling frequency) by applying a transformation such as a short-time discrete Fourier transformation, and stores them in memory area 2102 of the memory 2100 (step S 201 ).

Then, the frequency normalizing section 2121 of the signal separating section 2120 reads the frequency-domain mixed signals X q (f, τ) from memory area 2102 of the memory 2100 . After reading the frequency-domain mixed signals X q (f, τ), the frequency normalizing section 2121 normalizes a mixed-signal vector X(f, τ)=[X 1 (f, τ), . . . , X M (f, τ)] T consisting of those signals into a frequency-normalized vector X′(f, τ) that is independent of frequency f (step S 202 ). The generated frequency-normalized vectors X′(f, τ) are stored in memory area 2103 of the memory 2100 . Details of step S 202 will be described later.

Then, the norm normalizing section 2122 of the signal separating section 2120 read the frequency-normalized vectors X′(f, τ) from memory area 2103 of the memory 2100 and normalizes them into a norm-normalized vectors X″(f, τ) whose norm has a predetermined value (for example 1). The norm normalizing section 2122 then stores the generated norm-normalized vectors X″(f, τ) in memory area 2104 of the memory 2100 (step S 203 ). Details of this operation will be described later.

Then, the clustering section 2123 of the signal separating section 2120 reads the norm-normalized vectors X″(f, τ) from memory area 2104 of the memory 2100 , clusters them and generates clusters. The clustering section 2123 then stores cluster information C k identifying each cluster (information identifying the members X″(f, τ) of the k-th cluster (k=1, . . . , N), in memory area 2106 of the memory 2100 (step S 204 ). Details of this operation will be described later.

Then, the separated signal generating section 2124 of the signal separating section 2120 reads the cluster information C k and the reference value Q′ from memory areas 2106 and 2105 , respectively, of the memory 2100 . The separated signal generating section 2124 then uses the cluster information C k and the reference value Q′ to extract from memory area 2120 the Q′-th element X Q ′(f, τ) of the mixed-signal vector X(f, τ) corresponding to the norm-normalized vector X″(f, τ) belonging to the k-th cluster and generates a separated signal vector Y(f, τ) having the element as its k-th element Y k (f, τ). The separated signal generating section 2124 then stores the generated separated signal vector Y(f, τ) in memory area 2107 of the memory 2100 (step S 205 ). Details of this operation will be described later.

Finally, the time domain transforming section 2130 reads the separated signal vector Y(f, τ) from memory area 2107 of the memory 2100 and transforms each of its separated signal components Y k (f, τ) by using a transformation such as a short-time inverse Fourier transformation into a time-domain separated signal Y k (t) for each suffix k. The time domain transforming section 2130 then stores the transformed, time-domain separated signals y k (t) in memory area 2108 of the memory 2100 (step S 206 ).

Details of the operations will be described below.

[Details of Processing by the Frequency Normalizing Section 2121 and the Norm Normalizing Section 2122 ]

The frequency normalizing section 2121 and the norm normalizing section 2122 normalize all mixed-signal vectors X(f, τ)=[X 1 (f, τ), . . . , X M (f, τ)] T (f=0, f s /L, . . . , f s (L−1)/L) to norm-normalized vectors X″(f, τ) that are independent of frequency but dependent only on the positions of signal sources. This normalization ensures that each cluster formed by clustering at step S 204 corresponds only to a signal source. If this normalization is not properly performed, clusters are not formed. As described earlier, normalization in the sixth embodiment consists of frequency normalization and norm normalization. The frequency normalization is performed by the frequency normalizing section 2121 to normalize mixed-signal vectors X(f, τ) into frequency-normalized vectors X′(f, τ) that are independent of frequency. The norm normalization is performed by the norm normalizing section 2122 to normalize the frequency-normalized vectors X′(f, τ) into norm-normalized vectors X″(f, τ) whose norm has a predetermined value (1 in this example). These normalizations will be detailed below.

[Details of Processing by the Frequency Normalizing Section 2121 (Processing at Step S 202 )]

FIG. 35A is a flowchart illustrating details of processing at step S 202 shown in FIG. 34 . With reference to the flowchart, details of processing at step S 202 will be described below.

First, the control section 2140 ( FIG. 32 ) assigns 1 to parameter q and stores it in the temporary memory 2141 (step S 211 ). Then, the frequency normalizing section 2121 ( FIGS. 32 and 33 ) reads the parameters d, c, and Q described earlier from memory area 2105 of the memory 2100 , reads the element X q (f, τ) of the mixed-signal vector X(f, τ) corresponding to each (f, τ) from memory area 2102 , and reads parameter q from the temporary memory 2141 . The frequency normalizing section 2121 then calculates

[ Formula ⁢ ⁢ 50 ] X q ′ ⁡ ( f , τ ) =  X q ⁡ ( f , τ ) ⁢  exp ⁡ [ j ⁢ arg ⁡ [ X q ⁡ ( f , τ ) / X Q ⁡ ( f , τ ) ] 4 ⁢ ⁢ fc - 1 ⁢ d ] ( 60 )

and stores the result in memory area 2103 of the memory 2100 as the components of a frequency-normalized vector X′(f, τ)=[X′(f, τ), . . . , X M ′(f, τ)] T (step S 212 ). Here, arg[·] represents an argument and j represents an imaginary unit.

In particular, the first normalizing section 2121 a of the frequency normalizing section 2121 first normalizes the argument of each component X q (f, τ) of the mixed-signal vector X(f, τ) on the basis of a particular element X q (f, τ) of the mixed signal vector X(f, τ) by the following operation:

[Formula 51]

X q ′″( f ,τ)=| X q ( f ,τ)|exp{ j ·arg[ X q ( f ,τ)/ X Q ( f ,τ)]}  (61)

Then, the second normalizing section 2121 b of the frequency normalizing section 2121 divides the argument of each element X q ′″(f, τ) normalized by the first normalizing section 2121 a by a value 4fc −1 d proportional to frequency f as given below.

Then, the control section 2140 determines whether the value of parameter q stored in the temporary memory 2141 satisfies q=M (step S 213 ). If not q=M, the control section 2140 sets a calculation result q+1 as a new value of parameter q, stores it in the temporary memory 2141 (step S 214 ), and then returns to step S 212 . On the other hand, if q=M, the control section 2140 terminates step S 202 , and causes step S 203 to be executed.

[Details of Processing by the Norm Normalizing Section 2122 (Details of Step S 203 )]

FIG. 35B is a flowchart illustrating details of processing at step S 203 shown in FIG. 34 . With reference to the flowchart, processing at step S 203 will be detailed below.

The norm normalizing section 2122 ( FIGS. 32 and 33 ) reads the frequency-normalized vectors X′(f, τ)=[X 1 ′( f ,τ), . . . , X M ′(f, τ)] T corresponding to (f, τ) from memory area 2103 of the memory 2100 . The norm normalizing section 2122 then calculates their norms ∥X′(f, τ)∥ as

[ Formula ⁢ ⁢ 53 ]  X ′ ⁡ ( f , τ )  = ∑ q = 1 M ⁢ ( X q ′ ⁡ ( f , τ ) ) 2

and stores the frequency-normalized vectors X′(f, τ) and their norms ∥X′(f, τ)∥ in the temporary memory 2141 (step S 221 ).

Then, the norm normalizing section 2122 reads the frequency-normalized vectors X′(f, τ) corresponding to each (f, τ) and their norms ∥X′(f, τ)∥ from the temporary memory 2141 and calculates norm-normalized vectors X″(f, τ) as

X ″( f ,τ)= X ′( f ,τ)/∥ X ′( f ,τ)∥  (63)

(step S 222 ). The calculated norm-normalized vectors X″(f, τ) are stored in memory area 2104 of the memory 2100 and, with this, the processing at step S 203 ends.

The norm-normalized vectors X″ (f, τ) thus generated are independent of frequency and dependent only on the positions of the signal sources. Consequently, the norm-normalized vectors X″ (f, τ) form clusters. The reason why they form clusters will be described below.

[Reason Why Norm-Normalized Vectors X″(f, τ) form Clusters]

Because the sixth embodiment assumes the sparseness of source signals, each of the components X q (f, τ) of a mixed-signal vector X(f, τ) is proportional to (multiplied by a source signal S k (f, τ) which is a complex scalar) the frequency response H qk from the signal source k corresponding to the source signal p to a sensor q (X q (f, τ)=H qk (f, τ)·S k (f, τ)).

These source signals S k (f, τ) change with discrete time (that is, with phase). Of course, if the frequency f is the same, the relative value between the argument of a source signal S k (f, τ) observed at a sensor q and the argument of the source signal S k (f, τ) observed at reference sensor Q does not vary with discrete time.

As described above, the first normalizing section 2121 a of the frequency normalizing section 2121 normalizes the argument of each X q (f, τ) of a mixed-signal vector X(f, τ) on the basis of a particular element X Q (f, τ) of the mixed-signal vector X(f, τ) as a reference.

In this way, uncertainty due to the phase of the source signals S k (f, τ) is eliminated. Thus the argument of each element X q (f, τ) of the mixed-signal vector X(f, τ) that corresponds to the source signal p and sensor q is represented as a value relative to the argument of the element X Q (f, τ) of the mixed-signal vector X(f, τ) that corresponds to the source signal p and reference sensor Q (corresponding to reference value Q). In this case, the relative value corresponding to the argument of the element X Q (f, τ) is represented as 0.

The frequency response from the signal source k to the sensor q is approximated by using a direct-wave model without reflections and reverberations. Then, the argument normalize by the first normalizing section 2121 a described above will be proportional to both of the arrival time difference of a wave from a signal source k to sensors and the frequency f. Here, the arrival time difference is the difference between the time at which a wave from a signal source k reaches the sensor q and the time at which the wave reaches the sensor Q.

As described above, the second normalizing section 2121 b divides the argument of each component X q ′″(f, τ) normalized by the first normalizing section 2121 a by a value proportional to frequency f. Thus, the each element X q ′″(f, τ) is normalized to an element X q ′(f, τ) excluding the dependence of the argument on frequency. Consequently, the normalized elements X q ′(f, τ) will be dependent only on the arrival time difference of the wave from the signal sources k to the sensors. Here, the arrival time difference of the wave from the signal source k to the sensors is only dependent on the relative positions of the signal sources k, sensors q, and reference sensor Q. Therefore, for the same signal sources k, sensors q, and reference sensor Q, the elements X q ′(f, τ) have the same argument even if the frequency f differs. Thus, the frequency-normalized vector X′(f, τ) is independent of frequency f but is dependent only on the position of the signal source k. Therefore, clustering of norm-normalized vectors X″(f, τ) generated by normalization of the norms of the frequency-normalized vectors X′(f, τ) generates clusters each of which corresponds to the same signal source. In a real environment, the direct-wave model is not exactly satisfied because of the effects of reflections and reverberations. However, it provides a sufficiently good approximation as shown by experimental results, which will be given later.

The reason why the norm-normalized vectors X″(f, τ) form clusters will be described with respect to a model.

The impulse responses h qk (r) represented by Equation (1) given earlier is approximated by using a direct-wave (near-field) mixture model and represented in the frequency domain, as

[ Formula ⁢ ⁢ 54 ] H qk ⁡ ( f ) = γ ⁡ ( f ) d qk ⁢ exp ⁡ [ - j ⁢ ⁢ 2 ⁢ π ⁢ ⁢ fc - 1 ⁡ ( d qk - d Qk ) ] ( 64 )

where d qk is the distance between a signal source k and sensor q and y(f) is a constant dependent on frequency. The attenuation γ(f)/d qk is determined by the distance d qk and the constant γ(f), and the delay (d qk −d Qk )/c is determined by the distance normalized by using the position of sensor Q.

Assuming that the signals have sparseness, the following relationship holds at each time-frequency (f, τ).

X q ( f ,τ)= H qk ( f ,τ)· S k ( f ,τ)  (65)

From Equations (62), (63), (64), and (65), it follows that

As can be seen from this equation, the elements X q ″(f, τ) of the norm-normalized vector X″(f, τ) are independent of the frequency f and are dependent only on the positions of the signal sources k and sensors q. Therefore, when norm-normalized vectors are clustered, each of the clusters formed corresponds to the same signal source.

The same applies near-field and far-field mixed models that do not take attenuation of signals into consideration (as in the first embodiment).

It can be seen from Equation (66) that the value of parameter d is preferably d>d max /2 (where d max represents the maximum distance between the reference sensor corresponding to the element X Q ″(f, τ) and another sensor), more preferably d>d max , and yet more preferably d=d max , as with the first embodiment.

FIGS. 37 and 38 are complex planes illustrating the relationship between an element X q ″(f, τ) of a norm-normalized vector X″(f, τ) for each value of parameter d and its argument arg[X q ″(f, τ)]. The horizontal axis in the planes represents a real axis and the vertical axis represents an imaginary axis.

FIG. 37A shows a complex plane view showing the relationship when d max /2≧d. Here, from the definition of d max given above, the absolute value of d qk −d Qk is less than or equal to d max for any q and k. Therefore, if d max /2≧d, then (π/2)·(d qk −d Qk )/d≦−π and (π/2)·(d qk −d QK )/d≧π. Consequently, the arguments arg[X q ″(f, τ)] of X q ″ (f, τ) represented by Equation (66) can be distributed over the range beyond 2π, that is, α 1 ≦arg[X q ″(f, τ)]≦α 2 (α 1 ≦−π, α 2 ≧π). Accordingly, the arguments of elements X q ″(f, τ) of different norm-normalized vectors X″(f, τ) can be identical and therefore the different norm-normalized vectors X″(f, τ) can be clustered in the same cluster by the clustering described above. Therefore, it is desirable that d>d max /2. However, if there are no samples of norm-normalized vectors X″(f, τ) that correspond to the argument overlapping range, no problem arises even if d max /2≧d.

FIG. 37B shows a complex plane showing the case where d max /2<d<d max . In this case, the relationships −π<(π/2)·(d qk −D Qk )/d<−π/2 and π/2<(π/2)·(d qk −d Qk )/d<π are possible. Consequently, the arguments arg[X q ″(f, τ″)] of X q ″(f, τ) represented by Equation (66) can be distributed over the range β 1 ≦arg[X q ″ (f, τ)]≦β 2 (−π<β 1 <−π/2, π/2<β 2 <π). Accordingly, it is possible that the distance between elements of different norm-normalized vectors X″(f, τ) does not monotonically increase with increasing difference between the arguments of elements of different norm-normalized vectors X″(f, τ) in the ranges −π<arg[X q ″(f, τ)]<−π/2 and π/2<arg[X q ″(f, τ)]<π. This can degrade the accuracy of the clustering described above. Therefore it is desirable that d≧d max .

FIG. 38A is a complex plane of the case where d=d max and FIG. 38B is a complex plane of the case where d>d max . Here, if d>d max , the relation −π/2<(π/2)·(d qk −d Qk )/d<0, 0<(π/2)·(d qk −d Qk )/d<π/2 is possible. As a result, the arguments arg[X q ″(f, τ)] of X q ″(f, τ) represented by Equation (66) are distributed over the range γ 1 ≦arg[X q ″(f, τ)]≦γ 2 (−π/2<γ 1 <0 and 0<γ 2 <π/2) as shown in FIG. 38B . As the value d increases, the distribution range becomes narrower and clusters are distributed more densely in the narrow range. This degrades the accuracy of the clustering described above.

On the other hand, if d=d max , the relationship −π/2≦(π/2)·(d qk −d Qk )/d<0 and 0<(π/2)·(d qk −d Qk )/d≦π/2 are possible. Consequently, the arguments arg[X q ″(f, π)] of X q ″(f, τ) represented by Equation (66) are distributed over the range −π/2<arg[X q ″(f, τ)]≦π/2 as shown in FIG. 38A . In this case, clusters can be distributed over a range as wide as possible while maintaining the relationship in which the distance between elements of norm-normalized vectors X″(f, τ) monotonically increases as the difference between the arguments of the elements increases. Consequently, the accuracy of clustering can be improved in general. [End of the detailed description of (the processing by the frequency normalizing section 2121 and the norm normalizing section 2122 )]

[Details of Processing by the Clustering Section 2123 (Details of Step S 204 )]

As described earlier, the clustering section 2123 reads norm-normalized vectors X″(f, τ) from memory area 2104 of the memory 2100 and clusters them into M clusters. This clustering is performed so that the total sum U of the sums of squares U k of the members of the clusters (X″(f, τ)εC k ) and their centroids η k

[ Formula ⁢ ⁢ 56 ] U = ∑ k = 1 M ⁢ U k U k = ∑ X ″ ⁡ ( f , τ ) ∈ C k ⁢  X ″ ⁡ ( f , τ ) - η k  2

is minimized. The minimization can be performed effectively by using the k-means clustering described in Non-patent literature 6, for example. The centroid (center vector)η k of the cluster identified by cluster information C k can be calculated as

[ Formula ⁢ ⁢ 57 ] η k = ∑ X ′′ ⁡ ( f , τ ) ∈ C k ⁢ ⁢ X ′′ ⁡ ( f , τ ) /  C k   ∑ X ′′ ⁡ ( f , τ ) ∈ C k ⁢ ⁢ X ′′ ⁡ ( f , τ ) /  C k  

where |C k | is the number of members (norm-normalized vectors X″ (f, τ)) of the cluster identified by cluster information C k . While the distance used here is the square of the Euclidean distance, it may be the Minkowski distance, which is the generalized square of the Euclidean distance. [End of the detailed description of (the processing by the clustering section 2123 )]

[Details of Processing by the Separated Signal Generating Section 2124 (Details of Step S 205 )]

FIG. 36 is a flowchart illustrating details of processing at step S 205 shown in FIG. 34 . With reference to the flowchart, details of processing at step S 205 will be described below.

First, the control section 2140 ( FIG. 32 ) initializes the values of Y k (f, τ) for all values of parameter k (k=1, . . . , N) and time frequencies (f, τ) (all f and τ in a defined range) to 0 and stores them in memory area 2107 of the memory 2100 (step S 230 ).

The control section 2140 then assigns 1 to parameter k and stores it in the temporary memory 2141 (step S 231 ). Then the separated signal generating section 2124 ( FIGS. 32 and 33 ) reads the cluster information C k from memory area 2106 of the memory 2100 , extracts the members (norm-normalized vectors X″(f, τ)) of the k-th cluster identified by the cluster information C k , and stores them in the temporary memory 2141 (step S 232 ). The separated signal generating section 2124 then refers to the norm-normalized vectors X″(f, τ) stored at step S 232 in the temporary memory 2141 , reads mixed-signal vectors X(f, τ) in the time-frequencies (f, τ) corresponding to the norm-normalized vectors X″(f, τ) from memory area 2102 of the memory 2100 , and stores them in the temporary memory 2141 (step S 233 ). Then, the separated signal generating section 2124 reads the reference value Q′ from memory area 2105 of the memory 2100 and extracts (for each time-frequency (f, τ)) the Q′-th element X Q′ (f, τ) of the mixed-signal vector X(f, τ) stored at step S 233 . The separated signal generating section 2124 updates the values in memory area 2107 of the memory 2100 by setting the extracted element X Q′ (f, τ) as the k-th element Y k (f, τ) of the separated-signal vector Y(f, τ) (step S 234 ). That is, the separated signal generating section 2124 in this example extracts the element Y k (f, τ) as

Then, the control section 2140 determines whether the value of parameter k stored in the temporary memory 2141 satisfies k=N (step S 235 ). If not k=N, the control section 2140 sets a calculation result k+1 as a new value of parameter k, stores it in the temporary memory 2141 (step S 236 ), and then returns to step S 232 . On the other hand, if k=N, the control section 2140 terminates processing at step S 205 . [End of the detailed description of (processing by the separated signal generating section 2124 )]

<Experimental Results>

Results of experiments on sound source separation according to the sixth embodiment will be given below. In order to demonstrate the effects of the sixth embodiment, experiments on two types of signal separation were conducted.

In a first separation experiment, two sensors are used. Conditions of the experiment are shown in FIG. 39A . Three signal sources were used and English speech was emitted for 6 seconds through loudspeakers. Table in FIG. 39B shows the results of the experiments. As shown in the Table, the SIRs (signal-to-interference ratios) are improved. Greater numeric values represent better separation performances. Shown in the table are observations by the sensors before separation (InputSIR), results obtained using clustering with DOA (DOA (Previous)), and results obtained using the sixth embodiment (clustering using normalization) (Normalized obser. vector (Proposed)). The results reveal that, when two sensors are used, the method of the sixth embodiment can achieve signal separation with performance equivalent to the performance achieved by clustering using DOA alone.

In a second experiment, randomly arranged sensors are used. Experimental conditions are shown in FIG. 40A . In the experiment, four omnidirectional microphones (sensors) were nonlinearly arranged. Information indicating a maximum distance between microphones of 4 cm was all information provided to the separation system as to the arrangement of the sensors. Four signal sources were used to emit English speech for 6 seconds through loudspeakers. If DOAs were used in this arrangement of sensors and signal sources, a complicated process would have to be performed in which DOA of each sensor pair is estimated, clustering is performed for each sensor pair, and then the results of clustering at all sensor pairs are combined. The method of the sixth embodiment can achieve high separation performance as shown in the table in FIG. 40B without needing such a complicated combining operation. Furthermore, the second embodiment conducted under the conditions shown in FIG. 41A also showed high separation performance as shown in the table in FIG. 41B .

Features of the Sixth Embodiment

The features of the sixth embodiment are summarized below.

(1) Because all information obtained from mixed-signal vectors is used for clustering, information about all sensors can be effectively used and therefore the performance of signal separation is improved.

(2) Because precise information about the positions of sensors is not needed, a random arrangement of sensors can be used and sensor position calibration is not required.

<Variations>

The present invention is not limited to the sixth embodiment described above. For example, the first normalizing section 2121 a of the frequency normalizing section 2121 in the sixth embodiment normalizes the argument of each element X q (f, τ) of a mixed-signal vector X(f, τ) on the basis of a particular element X q (f, τ) of the mixed-signal vector X(f, τ) according to Equation (61). However, the first normalizing section 2121 a of the frequency normalizing section 2121 may normalize the argument of each element X q (f, τ) of a mixed-signal vector X(f, τ) on the basis of a particular element X q (f, τ) of the mixed-signal vector X(f, τ) according to any of the following equations.

X q ′″( f ,τ)=| X q ( f ,τ)|exp{ j ·(arg[ X q ( f ,τ)· X Q *( f ,τ)])} X q ′″( f ,τ)=| X q ( f ,τ)|exp{ j ·(arg[ X q ( f ,τ)]−arg[ X Q ( f ,τ)])} X q ′″( f ,τ)=| X q ( f ,τ)|exp{ j ·Ψ(arg[ X q ( f ,τ)/ X Q ( f ,τ)])}  [Formula 59]

Here, “*” is the complex conjugate of “·” and “Ψ{·}” is a function, preferably a monotonically increasing function from a viewpoint of clustering accuracy.

The frequency normalizing section 2121 may perform the frequency normalizing by using any of the following equations

[ Formula ⁢ ⁢ 60 ] ⁢ X q ′ ⁡ ( f , τ ) = ρ · arg ⁡ [ X q ⁡ ( f , τ ) / X Q ⁡ ( f , τ ) ] 4 ⁢ ⁢ fc - 1 ⁢ d ⁢ X q ′ ⁡ ( f , τ ) = ρ · arg ⁡ [ X q ⁡ ( f , τ ) · X Q * ( f , τ ) ] 4 ⁢ ⁢ fc - 1 ⁢ d X q ′ ⁡ ( f , τ ) = ρ · arg ⁡ [ X q ⁡ ( f , τ ) ] - arg ⁡ [ X Q ⁡ ( f , τ ) ] 4 ⁢ ⁢ fc - 1 ⁢ d ⁢ X q ′ ⁡ ( f , τ ) = ρ · Ψ ⁡ ( arg ⁡ [ X q ⁡ ( f , τ ) / X Q ⁡ ( f , τ ) ] ) 4 ⁢ ⁢ fc - 1 ⁢ d

instead of Equation (60). Here, ρ is a constant (for example ρ=1).

While the norm normalizing section 2122 in the sixth embodiment performs normalization so that the norm has a value of 1, it may perform normalization so that the norm has a predetermined value other than 1. Furthermore, the norm normalizing section 2122 is not provided and therefore norm normalization may be omitted. In that case, the clustering section 2123 clusters frequency-normalized vectors X′(f, τ). However, the norms of frequency-normalized vectors X′(f, τ) are not equal. Therefore, the clustering is performed based on whether vectors are similar only in direction, rather than both in direction and norm. This means evaluation based on the degrees of similarity. One example of the measure of similarity may be cosine distance

cos θ=| X′ H ( f ,τ)·η k |/(∥ X ′( f ,τ)∥·∥η k ∥)

where θ is the angle between a frequency-normalized vector X′(f, τ) and the vector of the centroid η k . If the cosine distance is used, the clustering section 2123 generates a cluster that minimizes the total sum of cosine distances

U i =Σx p ′( f ,τ)ε C i |X p ′ H ( f ,τ)·η i |/(∥ X p ′( f ,τ)∥·∥η i ∥)   [Formula 61]

Here, the centroid η k is the average among the members of each cluster.

The reference values Q and Q′ given above may or may not be equal.

The same value of parameter d may be set for all sensors q or different values of parameter d may be set for different sensors q. For example, the distance between a reference sensor and a sensor q may be set as the value of parameter d for the sensor q.

Furthermore, the separated signal generating section 2124 may generate, instead of

[ Formula ⁢ ⁢ 62 ] Y k ⁡ ( f , τ ) = { ⁢ X Q ′ ⁡ ( f , τ ) X ′′ ⁡ ( f , τ ) ∈ C k ⁢ 0 otherwise

the following binary mask

[ Formula ⁢ ⁢ 63 ] M k ⁡ ( f , τ ) = { 1 X ′′ ⁡ ( f , τ ) ∈ C k 0 otherwise

and obtain the k-th element Y k (f, τ) of a separated signal vector Y(f, τ) as

Y k ( f ,τ)= M k ( f ,τ) X Q′ ( f ,τ)

While a Fourier transformation or an inverse Fourier transformation is used for transformation between the frequency domain and the time domain in the embodiments described above, a wavelet transformation, DFT filter bank, polyphase filter bank or the like may be used for the transformation (for example see R. E. Crochiere, L. R. Rabiner, “Multirate Digital Signal Processing.” Eaglewood Cliffs, N.J.: Prentice-Hall, 1983 (ISBN 0-13-605162-6). The operations described above may be performed in time sequence in accordance with the description or may be performed in parallel or separately, depending on the throughput capacity of the apparatus that performs the operations. It will be understood that any other modifications may be made without departing from the spirit of the present invention.

If any of the embodiments described above is implemented by a computer, operations to be performed by each apparatus are described by a program. The processing functions described above are implemented on the computer by executing the program.

The program describing these processing operations can be recorded on a computer-readable recording medium. The computer-readable medium may be any medium such as a magnet recording device, an optical disk, magneto-optical recording medium, or a semiconductor memory. In particular, the magnetic recording device may be a hard disk device, a flexible disk, or a magnetic tape; the optical disk may be a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable/RW (ReWritable); the magneto-optical recording medium may be an MO (Magneto-Optical disc); and the semiconductor memory may be an EEP-ROM (Electronically Erasable and Programmable-Read Only Memory).

The program may be distributed by selling, transferring, or leasing a removable recording medium such as a DVD or a CD-ROM, for example, on which the program is recorded. Alternatively, the program may be distributed by storing it in a storage device of a server computer beforehand and transmitting it from the server computer to another computer via a network.

In an alternative embodiment, a computer may directly read the program from a removable recording medium and execute processing according to the program, or the computer may execute processing according to the program each time the program is transmitted from a server to the computer. Alternatively, the computer may execute the processing described above using an ASP (Application Service Provider) service in which the program itself is not transmitted from a server computer to the computer, instead, the computer implements the processing by obtaining only instructions of the program and the results of execution of the instructions. The program in this mode includes information that is made available for processing by computer and is a quasi-program (such as data that are not direct instructions to a computer but defines processing to be performed by the computer).

While a given program is executed on a computer to configure the present embodiments, at least part of the processing described above may be implemented by hardware.