Noise reduction

This invention relates to estimating features of a signal, particularly for the purpose of reducing noise in the signal. The features could be noise power and gain. The signal could be an audio signal.

There are many types of devices that detect and process speech signals. Examples include headsets and mobile phones. In those devices it is often desired to reduce the noise in the detected signal in order to more accurately represent the speech component of the signal. For instance, in a mobile phone or a headset any audio that is detected by a microphone may include a component representing a user's speech and a component arising from ambient noise. If that noise can be removed from the detected signal then the signal can sound better when it is played out, and it might also be possible to compress the signal more accurately or more efficiently. To achieve this, the noise component of the detected audio signal must be separated from the voice component.

If a speech signal s(n) is corrupted by additive background noise v(n), the resulting noisy speech signal d(n) can be expressed in the time domain as:

d ( n )= s ( n )+ v ( n )  (1)

The objective of noise reduction in such a situation is normally to estimate v(n) and subtract it from d(n) to find s(n).

One algorithm for noise reduction operates in the frequency-domain. It tackles the noise reduction problem by employing a DFT (discrete Fourier transform) filter bank and tracking the average power of quasi-stationary background noise in each sub-band from the DFT. A gain value is derived for each sub-band based on the noise estimates, and those gain values are applied to each sub-band to generate an enhanced time domain signal in which the noise is expected to be reduced. FIG. 1 illustrates this algorithm by a block diagram. The incoming signal d(n) is received at 1. It is applied to a series of filters 2 , each of which outputs a respective sub-band signal representing a particular sub-band of the incoming signal. Each of the sub-band signals is input to a downsampling unit 3 which downsamples the sub-band signal to average its power. The outputs of the downsampling units 3 form the output of the analysis filter bank (AFB) 5 . Those outputs signals are noisy signals D k (k=0 . . . M−1). Each of those signals is subsequently multiplied by G oms,k in a multiplication unit 6 . G oms,k is an estimated gain value that will be discussed in more detail below. Then the enhanced time domain signal is obtained by passing the multiplication results through a synthesis filter bank (SFB). In the SFB 7 upsampling units 8 upsample the outputs of the multiplication units, the outputs of the upsampling units are applied to respected synthesis filters 9 which each re-synthesise a signal representing the respective sub-band, and then the outputs of the synthesis filters are added to form the output signal.

In general, it can be assumed that the speech signal and the background noise are independent, and thus the power of the noisy speech signal is equal to the power of the speech signal plus the power of background noise in each sub-band k

| D k | 2 =S k | 2 +|V k | 2 .  (2)

If the noise power is known then an estimate of the speech power can be got from:

| S k | 2 =|D k 2 −|V k | 2 ,  (3)

It is necessary to estimate the gain in order to generate the signals G oms,k . One of the most widely used methods of estimating gain is by means of the optimal Wiener filter gain, which is computed as

The estimated clean speech signal in each sub-band, Ŝ k , is then simply derived as

Ŝ k =G wiener,k ·D k .  (5)

It can be identified that the estimation of noise power (|V k | 2 ) and gain (G oms ) is crucial to the success of the algorithm. Unfortunately, obtaining reliable estimates of these has shown to be extremely difficult in the past due to the high complexity of various noisy environments. Many algorithms perform well in one situation but fail in other situations. Since the nature of the environment is not normally known in advance, and may change as a user moves from place to place, many algorithms provide inconsistent and unsatisfactory results.

It would therefore be valuable to have an improved mechanism for estimating noise power in a signal.

According to aspects of the present invention there are provided signal processing apparatus and methods as set out in the accompanying claims.

The present invention will now be described by way of example with reference to the accompanying drawings, in which:

FIG. 1 is a block diagram showing a mechanism for reducing noise in a signal;

FIG. 2 is a block diagram showing a mechanism for estimating noise power in a signal;

FIG. 3 shows a state machine for using minimum statistics; and

FIG. 4 shows a state machine for determining the value of an over-subtraction factor.

The system described below estimates noise in an audio signal by means of an adaptive system having cascaded controller blocks.

This example will be described in the context of a device for estimating noise in a source audio signal. FIG. 2 shows the general logical architecture that will be employed. The source audio signal d(n) will be applied to an analysis filter bank (AFB) 10 analogous to that shown in FIG. 1 and to a harmonicity estimation unit 11 which generates an output dependent on the estimated harmonicity of the source signal. The outputs of the analysis filter bank 10 and the harmonicity estimation unit 11 are provided to a statistical analysis unit 12 which generates minimum statistics information. The statistical analysis unit processes the output of the AFB in a manner that is dependent on the output of the harmonicity estimation unit. The outputs of the analysis filter bank 10 and the statistical analysis unit are applied to an adaptive noise estimation unit 13 which adaptively estimates the noise in each sub-band of the signal by processing the output of the AFB in a manner that is dependent on the output of the statistical analysis unit.

Let a noise power estimate be denoted by P k (l), where k is the sub-band index and l is the frame index of the data frame under consideration after processing by the analysis filter bank 10 with downsampling rate L. As shown by FIG. 2 , P k (l) is obtained after the input signal passes through the AFB and though the adaptive noise estimation unit 13 . In parallel with the AFB are the modules 11 and 12 . The dashed arrows in FIG. 2 indicate that the outputs of modules 11 and 12 control the operation of the units to which they are input.

For better illustration, in the following the operation of the modules 10 to 13 will be described in reverse order.

Adaptive Noise Estimation Module

Noise power P k (l) is commonly estimated by applying a first-order IIR filter to the noisy signal power:

P k ( l )= P k ( l− 1)+α(| D k ( l )| 2 −P k ( l− 1)),  (6)

where the parameter α is a constant between 0 and 1 that sets the weight applied to each frame, and hence the effective average time.

Adaptive noise estimation is achieved by weighting α in equation (6) dynamically with a speech absence probability (SAP) model. That model is described below.

Let H 0 be the hypothesis of speech absence; then the speech absence probability (SAP) given an input signal in the frequency domain (D) is p(H 0 |D). For simplicity, time and frequency indices will be ignored in the description below. Applying Bayes' rule one obtains:

p ⁡ ( H 0 | D ) = p ⁡ ( D | H 0 ) ⁢ p ⁡ ( H 0 ) p ⁡ ( D ) .

⁢ Assuming ( 7 ) p ⁡ ( H 0 ) = λ , ( 8 )

where λ is a constant between 0 and 1, inclusive, then for a complex Gaussian distribution of DFT coefficients (D), we have

p ⁡ ( D ) = 1 πσ D 2 ⁢ exp ⁡ ( -  D  2 σ D 2 ) ,

⁢ and ( 9 ) p ⁡ ( D | H 0 ) = 1 π ⁢ ⁢ P ⁢ exp ⁡ ( -  D  2 P ) , ( 10 )

where σ D 2 is the variance of D. (See Vary, P.; Martin, R. Digital Speech Transmission. Enhancement, Coding and Error Concealment , John Wiley-Verlag, 2006; Y. Ephraim and D. Malah, “Speech enhancement using a minimum mean-square error log-spectral amplitude estimator,” IEEE Trans. Acoustics, Speech and Signal Processing, vol. ASSP-33, pp. 443-445, 1985; and I. Cohen, “Noise Spectrum Estimation in Adverse Environments: Improved Minima Controlled Recursive Averaging,” IEEE Trans. Speech and Audio Processing, vol. 11, pp. 466-475, September 2003).

Combining equations 7 to 10 gives the conditional speech absence probability as being:

By substituting σ D 2 with instantaneous signal power |D| 2 , and also adding additional constraints to differentiate between different conditions, equation 11 can be re-written as

q k ⁡ ( l ) = {  D k ⁡ ( l )  2 P k ⁡ ( l ) ⁢ exp ⁡ ( 1 -  D k ⁡ ( l )  2 P k ⁡ ( l ) ) ⁢ λ , if ⁢ ⁢  D k ⁡ ( l )  2 > P k ⁡ ( l ) λ , otherwise ( 12 )

and the noise power estimation becomes

P k ( l )= P k ( l− 1)+α q k ( l )(| D k ( l )| 2 −P k ( l− 1)).  (13)

It can be observed that q k (l) reaches λ only when |D k (l)| 2 is equal to P k (l), and approaches 0 when their difference increases. This feature allows smooth transitions to be tracked but prevents any dramatic variation from affecting the noise estimate. Note that setting q k (l)to λ when |D k (l)| 2 is smaller than P k (l) enables full speed noise adaptation which can preserve weak speech segments better as it reduces the weight of previous noise estimates. The drawback of this is the noise estimates are biased toward lower values that results in less noise reduction. This can be mitigated in a manner described below.

The SAP model in equations 12 is derived from the energy ratio between a noisy speech signal and estimated noise within each individual frequency band. It does not take advantage of the following known facts:

Voiced speech signals usually have a harmonic structure. Speech signals have a distinct formant structure.

By supposing that noise under consideration does not have those structures characteristic of speech, a more effective SAP model can be derived to detect speech or noise. One option is to modify equations 12 to incorporate cross-band averaging, in the following way:

R k ⁡ ( l ) = ∑ j = k - b ⁡ ( k ) k + b ⁡ ( k ) ⁢  D j ⁡ ( l )  2 ∑ j = k - b ⁡ ( k ) k + b ⁡ ( k ) ⁢ P j ⁡ ( l ) , ( 14 ) q k ⁡ ( l ) = R k ⁡ ( l ) ⁢ exp ⁡ ( 1 - R k ⁡ ( l ) ) ⁢ λ , ( 15 )

where b(k) is a predefined bandwidth value for sub-band k.

Such cross-band averaging results in greater variance reduction on noise than on speech, and makes the SAP model more robust. However, excessive averaging (i.e. a value of b(k) that is too large) will reduce both frequency and time resolution, which can cause significant speech distortion. To avoid this bandwidth values should be selected to be in-keeping with the formants present in speech, for example:

(1) By increasing bandwidth values with increasing frequency, since formant bandwidth generally increases with formant frequency. (2) By using relatively narrower bandwidth for the regions of the first and second formants, since these regions are more important to speech intelligibility.

Speech absence probability can alternatively be estimated by other voice activity detection algorithms, conveniently those that output SAP based on input signal power information.

Statistical Analysis Module

Adaptive noise estimation performed as described above may need a long time to converge when there is a sudden change of noise floor. One possible solution is to use minimum statistics to correct noise estimation. (See Rainer Martin, “Noise power spectral density estimation based on optimal smoothing and minimum statistics,” IEEE Transactions on speech and audio processing, vol. 9, no. 5, pp. 504-512, July 2001; Myron J. Ross, Harry L. Shaffer, Andrew Cohen, Richard Freudberg).

The approach employed in the present system essentially involves searching for a minimum value either:

(a) in the time domain; or (b) in the frequency domain within a time frame,

and then using this value or its derivative as the noise estimates.

In the present system, minimum statistics are used to control the adaptive noise estimator, whereby the requirement for high frequency resolution can be greatly relaxed. Specifically, instead of performing minimum tracking in each sub-band, we group frequency bins into several subsets and obtain one minimum value for each subset. The benefit of grouping is two-fold: (1) it reduces system complexity and resource cost; and (2) it smoothes out unwanted fluctuation. Without loss of generality, we split the spectrum into two groups in our implementation, which span low frequency and high frequency regions, respectively. More groups could be used, and non-adjacent portions of the frequency spectrum could be combined in a single group. For each group, a fixed length FIFO (first-in first-out) queue is formed by taking the summation of noisy signal power (|D k (l)| 2 ) for each frame. Finally one minimal value is identified for each queue.

Minimum statistics are used in the following way to aid adaptive noise estimation. Let P min,g (l) be the minimum power value for group g at frame index l determined in the manner described above, and let P sum,g (l) represent the total estimated noise power for group g at frame l. Then a correction factor C is derived as

The control of noise estimation using minimum statistics is realized through applying this correction factor to the noise estimates P k (l).

To take further advantage of minimum statistics information, a more complex scheme can be used. The range of C {C≧0} can be divided into four zones by defining two threshold values T 1 and T 2 , where T 1 <1<T 2 . Then a state machine is implemented as shown in FIG. 3 .

When the minimum P min,g (l) is only slightly lower than estimated noise power P sum,g (l) as in state 2 (T 1 ≦C≦1), nothing needs to be done because this is fully expected. However, if the minimum value is significantly smaller than noise estimate as in state 1 (C<T 1 ) then a correction is triggered. State 1 corresponds to a condition where noise becomes mistakenly adapted to speech level or there is a sudden drop of noise floor. To avoid over-adjustment, the correction factor C is normalized by T 1 so that the corrected noise estimates are still higher than the minimum value. When P min,g (l) is greater than P sum,g (l) as in state 3 (1<C≦T 2 ), simple correction is applied as there might be a sudden jump of noise floor and our noise estimate is lagging behind. Special treatment is needed when the minimum value (P min,g (l)) is significantly higher than the noise estimate (P sum,g (l)) as in state 4 (C>T 2 ). A plain correction of multiplying by the correction factor may run into problems when there is a substantial spectrum mismatch between the old noise floor and the new noise floor. It may take very long time to converge to the new noise spectrum. Or, even more problematically, narrow band noise could be produced which might well create annoying audio artefacts. This is addressed in the state machine of FIG. 3 by resetting noise estimates to white spectrum for each group, as shown in equation 18. This employs the property that when the noise floor change is too extreme using the evenly distributed spectrum may well result in quick convergence.

Harmonicity Module

The minimum-search window duration has a crucial impact on noise estimation. A short window allows faster response to noise variation but may also misclassify speech as noise when continuous phonation is longer than the window length. A long window on the other hand will slow down noise adaptation. One approach is to define an advantageous window length empirically, but this may not suit a wide range of situations. Instead, the present system employs a dynamic window length which can vary during operation. In this example the window length is controlled by speech harmonicity (periodicity).

There are many ways to determine harmonicity of speech. AMDF (Average Magnitude Difference Function) is one method, and is described in Harold J. Manley; Average magnitude difference function pitch extractor, IEEE Trans. Acoust., Speech, Signal Processing, vol. 22, pp. 353-362, October 1974. A variant of AMDF is CAMDF (Cross Average Magnitude Difference Function). CAMDF has been found to be relatively efficient and to provide relatively good performance.

For a short-term signal x(n) {n:0 . . . N−1} CAMDF can be defined as below:

where τ is the lag value that is subject to the constraint 0<τ≦N−U.

One representation of harmonicity based on CAMDF can simply be the ratio between its minimum and maximum:

A harmonicity value is conventionally used directly to determine voicing status. However, its reliability degrades significantly in a high noise environment. On the other hand, under medium to high SNR conditions, harmonicity offers some unique yet important information previously unavailable to adaptive noise estimation and minimum statistics which exploit mostly energy variation patterns. The present system uses harmonicity to control the manner of operation of the statistical analysis module. Specifically, when a frame is classified as voiced by the harmonicity function, it is skipped by the minimum statistics calculation. This is equivalent to lengthening the minimum search window duration when speech is present. As a result, the default search duration can be set relatively short for fast noise adaptation.

The harmonicity detector/module can be alternatively implemented through other pitch detectors described in the literature, for example by auto-correlation. However, it is preferable to use a simpler method than fully-fledged pitch detection since pitch detection is computationally intensive. Alternatives include determining any one or more of harmonicity, periodicity and voicing and/or by analysing over a partial pitch range. If voicing is used then the detector need not perform any pitch detection.

Instant Noise Estimation Using Fourier Transform of AMDF and Variable Start Minima Search [Zhong Lin; Goubran, R.; Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP apos;05). Volume 1, Issue, Mar. 18-23, 2005 Page(s): 161-164 discloses a speech processor that employs a speech detector based on Fourier Transform of AMDF that running in parallel with Variable Start Minima Search. Such a parallel approach—unlike the cascading approach described herein—increases the system's sensitivity to speech detector failures and can be computationally less efficient.

Hybrid Gain from Wiener Filter with Over-Subtraction and MMSE-LSA

Gain calculated based on the Wiener filter in equation 4 often results in musical noise. One of the commonly used solutions is to use over-subtraction during gain calculation as shown below.

G wiener , k ⁡ ( l ) = max ⁡ ( 1 - β ⁢ ⁢ P k ⁡ ( l )  D k ⁡ ( l )  2 , 0 ) , ( 21 )

where β is the over-subtraction factor.

As mentioned earlier, the noise estimate P k (l) in the present system can be found to be biased toward lower values. Thus, using over-subtraction also compensates noise estimation to achieve greater noise reduction.

In the present system, an adaptive over-subtraction scheme is used, which is based on the SAP obtained as described above. First, let β min and β max be the minimum and maximum over-subtraction values, respectively. Then in a similar manner to the analysis performed in the statistical analysis module described above, and ignoring time and frequency subscripts for simplicity, we divide the range of speech absence probability q into three zones by defining two threshold values Q S and Q N such that 0<Q S <Q N <1. This represents a crude categorization of SAP into speech only, speech mixed with noise, and noise only states, respectively. Finally we use a state machine to determine the value of over-subtraction factor β. The state machine is illustrated in FIG. 4 .

In state 1 (speech only) or state 3 (noise only), β is simply set to the pre-determined minimum or the maximum over-subtraction values respectively. In state 2 which corresponds to a mixed speech and noise condition, β is calculated by linear interpolation between β min and β max based on SAP q. With properly selected threshold values, over-subtraction can effectively suppress musical noise and achieve significant noise reduction overall.

To further suppress musical noise, additional processing is applied to the instantaneous gain G wiener,k (l).

Because noise is a random process, the true noise power at any instance varies around the noise estimate P k (l). When G wiener,k (l) is much larger than P k (l), the fluctuation of noise power is minor compared to |D k (l)| 2 , and hence G wiener,k (l) is very reliable and its normalized variance is small. On the other hand, when |D k (l)| 2 approximates P k (l), the fluctuation of noise power becomes significant, and hence G wiener,k (l) is unreliable and its normalized variance is large. If G wiener,k (l) is left without further smoothing, the large normalized variance in low SNR periods would cause musical or watering artefacts. However, if a constant average rate is used to suppress these artefacts, it would cause over smoothing in high SNR periods and thus results in tonal or ambient artefacts. To achieve the same normalized variation for the gain factor, the average rate needs to be proportional to the square of the gain. Therefore the final gain factor G k (l) is computed by smoothing G wiener,k (l) with the following algorithm:

G k ( l )= G k ( l− 1)+(α G ·G 0,k 2 ( l ))( G weiner,k ( l )− G k ( l− 1)),  (23)

G 0,k ( l )= G k ( l− 1)+0.25( G wiener,k ( l )− G k ( l− 1)),  (24)

where α G is a time constant between 0 and 1, and G 0,i (k) is a pre-estimate of G k (l) based on the latest gain estimate G k (l−1) and the instantaneous Wiener gain G 0,k (l). Using a variable average rate G 0,k 2 (l), and specifically one based on a pre-estimate of the moderated Wiener gain value, to smooth the Wiener gain can help regulate the normalized variance in the gain factor G k (l)

It can be observed that G k (l) is averaged over a long time when it is close to 0, but is with very little average when it approximates 1. This creates a smooth noise floor while avoiding generating ambient-sounding (i.e. thin, watery-sounding) speech.

While over-subtraction and gain smoothing create a smooth noise floor and achieve significant noise reduction, they could also cause speech distortion, particularly on weak speech components. To improve voice quality, we choose MMSE-LSA gain function described in Ephraim and D. Malah to replace equation 21 for certain conditions which will be specified later.

The formulation of MMSE-LSA is described below.

First, define:

γ k ⁡ ( l ) ⁢ = Δ ⁢  D k ⁡ ( l )  2 P k ⁡ ( l ) , ( 25 ) ξ k ⁡ ( l ) ⁢ = Δ ⁢ S ^ k ⁡ ( l ) P k ⁡ ( l ) , ( 26 )

where γ is the a posteriori SNR, and ξ is the a priori SNR.

Then the MMSE-LSA gain function is:

In MMSE-LSA, a priori SNR ξ is the dominant factor, which enables filter to produce less musical noise and better voice quality. However, because of the diminishing role of a posteriori SNR γ, on which the over-subtraction can be applied, the noise reduction level of MMSE-LSA is limited. For this reason the present system only uses MMSE-LSA for speech dominant frequency bands of voiced frames. This is because on those frames: (1) speech quality matters most, and (2) less noise reduction may be tolerable as some noise components might be masked by stronger speech components.

Results

Tests using the system described above have indicated that the system can achieve over 20 dB noise reduction while preserving high voice quality. The system has been found to perform well from quiet to high noise conditions. It has also been found to have a fast convergence time of less than 0.5 seconds in some typical environments. These results place it among the best currently available algorithms for single microphone noise reduction performance.

The system described above can be used to estimate noise power and/or gain for use in a noise reduction system of the type shown in FIG. 1 , or in another such system, or for other purposes such as identifying an environment from its noise characteristics.

The system described above can be implemented in any device that processes audio data. Examples include headsets, phones, radio receivers that play back speech signals and stand-alone microphone units.

The system described above could be implemented in dedicated hardware or by means of software running on a microprocessor. The system is preferably implemented on a single integrated circuit.

The inventors hereby disclose in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The inventors indicate that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.