Method and apparatus for objective sound quality measurement using statistical and temporal distribution parameters

The present invention relates generally to speech quality measurement and, more particularly, to speech quality measurement of voice transmitted over a packet network.

Perceived speech quality assessment has traditionally been performed using subjective testing, which involves considerable time, effort and resources. Subjective tests are carried out by having a number of listeners come in and listen to a set of speech files and rate them on a subjective scale. Objective speech quality metrics try to estimate the perceived speech quality by comparing the original and distorted speech signals.

Traditional objective measures such as Signal to Noise Ratio (SNR) do not provide a good estimate of subjective quality, especially when sophisticated low bit rate speech coding techniques are used. An auditory model can be used to perceptually weight the distortion between the original and the test signals, to compute the perceptually significant distortion.

Other methods using a perceptual model compute a weighted average of the frame based perceptually weighted distortion measure to compute the objective quality score. One such method is PSQM (Perceptual Speech Quality Measure) which is used in ITU-T standard P.861. This method uses a perceptual model to map the original and test speech signals onto a psychophysical representation to compute a “noise disturbance” for each frame of speech. The PSQM score is computed as a weighted average of the “noise disturbance” where silence frames and speech frames are given different weights. The “noise disturbance” of PSQM is an example of a frame based perceptual distortion.

A PSQM test system 100 is shown in FIG. 1. A sound source 10 generates a series of sound sample frames x[n] which are input to a signal processor 20 . The signal processor 20 processes the sound sample frames x[n] and outputs a series of test or coded sound frames y[n]. The series of sound sample frames x[n] and the series of coded sound frames y[n] are then input to PSQM processor 30 which processes the two series and generates PSQM parameters which evaluate the quality of the coding performed by the signal processor 20 .

FIG. 2 is a block diagram which describes the PSQM algorithm performed by the PSQM processor 30 . Within PSQM, the physical signals constituting the source and test speech, x[n] and y[n] respectively, are mapped onto psychophysical representations that match the internal representations of the speech signals (i.e. the representations inside our heads) as closely as possible. These internal representations make use of the psychophysical equivalents of frequency (critical band rates) and intensity (Compressed Sone). Masking is modeled in a simple way: masking is taken into account only when two time-frequency components coincide in both the time and frequency domains.

Within the PSQM approach, the quality of the test speech is judged on the basis of differences in the internal representation. This difference is used to calculate the noise disturbance as a function of time and frequency. In PSQM, the average noise disturbance is directly related to the quality of test speech. The PSQM approach is discussed in detail in ITU Recommendation P.861 “Methods for Objective and Subjective Assessment of Quality”.

A sound quality evaluation processor, according to the present invention, includes a comparator and a sequence processor. The comparator has first and second inputs and an output. The first input is configured to receive a sequence of sound sample frames and the second input is configured to receive a sequence of test sound frames. The comparator is configured to compare each frame of the sequence of test sound frames to a corresponding one of the sequence of sound sample frames in order to generate a sequence of distortion measure values at the output of the comparator. The sequence processor has first and second inputs and a first output. The first input is configured to receive the sequence of distortion measure values from the comparator and the second input is configured to receive a temporal outlier distortion threshold value. The sequence processor detects temporal-outlier sequences (TOSs) in the distortion measure values that are greater than the temporal outlier distortion threshold value. An average TOS length is then computed for output at the first output of the sequence processor.

The sound quality evaluation processor, according to the present invention, can also include an outlier processor having a first input configured to receive the sequence of distortion measure values from the comparator and a second input being configured to receive a perceptual outlier distortion threshold value. The outlier processor detects each perceptual outlier frame having a distortion measure value greater than the perceptual outlier distortion threshold value. The number of perceptual outlier frames is divided by the number of distortion measure values to obtain a percent of perceptual outliers output at the first output of the outlier processor.

The features and advantages of the invention will become more readily apparent from the following detailed description of a preferred embodiment of the invention which proceeds with reference to the accompanying drawings.

FIG. 1 is a functional block diagram illustrating a conventional sound quality test system.

FIG. 2 is a block diagram of a prior art PSQM algorithm performed by the PSQM processor of FIG. 1 .

FIG. 3 is a functional block diagram illustrating a prior art sound quality test system according to the present invention.

FIG. 4 is a functional block diagram of the statistical and temporal processor of FIG. 3 .

A test system 300 including a statistical and temporal processor 400 according to the present invention is shown in FIG. 3 . Similar to the PSQM system 100 of FIG. 1, sound source 10 generates the series of sound sample frames x[n] which are input to signal processor 20 . The signal processor 20 processes the sound sample frames x[n] and outputs the series of coded or test sound frames y[n]. The series of sound sample frames x[n] and the series of test sound frames y[n] are then input to statistical and temporal processor 400 which processes the two series and generates statistical and temporal parameters which evaluate the quality of the coding performed by the signal processor 20 . The statistical and temporal parameters produced by statistical and temporal processor 400 are input to quality score processor 320 which combines the parameters to calculate an objective sound quality score M.

The score M can then be used to select a device suitable for sound transmission. For instance, the objective sound quality score values for a number of transmission channels can be analyzed by a selection processor to choose the best transmission channel to carry a voice connection.

The present invention is directed toward a method and apparatus for objectively measuring speech quality over a channel or system whose characteristics vary with time or with input sound. Other objective sound quality measures use a weighted average of “frame based perceptual distortion”. The present invention uses statistical and temporal distribution parameters to obtain an improved objective measure.

Note that the signal processor 20 of FIGS. 1 and 3 performs a coder/decoder function that can include network or transmission equipment, such as a network channel. The sample sounds are encoded, transmitted over the channel and then decoded to obtain the test sound frames which reflect the conditions present on the channel. Thus, the present invention permits the objective sound quality over a transmission channel, such as a packet network, to be estimated under different network conditions, such as varying network load, jitter, and packet loss rate.

Conventional methods, such as PSQM, typically use an average of frame based perceptually weighted distortion to estimate speech quality. The conventional approach works well for cases in which the channel or system introducing the distortion is reasonably invariant. However, in cases where the distortion varies with time, such as in a channel with frame erasures, the average distortion is not a good indicator of perceived quality.

In cases where the distortion varies, the perceived quality is also dependent upon the statistical and temporal distribution of the distortion. Take the case of a transmission system which uses a high rate voice coder to achieve very low distortion. Even if a few frames are lost, the average distortion remains fairly low even though the perceived quality is poor due to the lost frames.

The present invention uses statistical and temporal analysis of frame based perceptual distortion to compute objective speech quality parameters. The frame based perceptual distortion measure is analyzed to compute the average value as well as the variance and the number of outliers. Here, an outlier is defined as a frame with distortion high enough to be perceptually disruptive. The number of outliers is the number of frames for which the distortion is greater than a predetermined threshold. The percentage of outliers equals:

(the number of outliers)*(100)/(total number of frames).

Temporal analysis is used to find lengths of sequences of frames with high distortion. A long sequence of frames with high distortion is perceptually more disruptive then a single frame with high distortion. A long sequence of outliers can be caused by bursty frame loss in a channel. The distortion threshold used in temporal analysis need not be the same as that used to compute the number of outliers above.

FIG. 4 is a block diagram of the statistical and temporal processor 400 of FIG. 3 . The series of sound sample frames x[n] and the series of test sound frames y[n] are input to comparator 410 which generates a series of distortion measure frames d[I], I=1 . . . N, where d[I] is the distortion measure between corresponding frames of the sound sample and test sound signals for N frames of each. An example of the comparator 410 is the perceptual technique used in the PSQM algorithm described in ITU-T P.861. The distortion frames d[I] are then stored in store 420 for processing by distortion processor 430 , outlier processor 440 and sequence processor 450 , which generate statistical and temporal parameters estimating the quality of the test sound frames y[n] produced by signal processor 20 .

Distortion processor 430 produces two objective statistical measures of sound quality: an average perceptual distortion measure D_avg and a variance of distortion D_var. The average perceptual-distortion measure D_avg is determined using equation (1) as follows: D_avg = 1 N  ∑ I = 1 N     d  [ I ] ( 1 )

Variance of perceptual distortion measure D_var is a statistical measure of how much the distortion in the test sound frames y[n] varies over the sequence of N frames. D_var is determined by distortion processor 430 according to equation (2) below: D_var = 1 N  ∑ I = 1 N     d  [ I ] 2 - D_avg 2 ( 2 )

Outlier processor 440 generates two temporal measures of sound quality: a number of outlier frames N_o and a percent of outlier frames P_o. The number of oulier frames N_o is determined by comparing each of the sequence of distortion measures d[I] to predetermined outlier threshold value D 1 _th. D 1 _th is selected to be an approximation of the level of distortion which a listener is likely to find annoying, as determined from subjective testing for example. Frames that have greater distortion than D 1 _th are considered outlier frames.

The total count of outlier frames in the sequence of N frames is N_o. From the number of frames N and the number of outlier frames N_o, the percentage of outlier frames P_o is obtained. These measures reflect the number and percentage, respectively, of frames produced by the signal processor 20 that have a perceptually disruptive level of distortion. The algorithm performed by outlier processor 440 can be described as follows:  N_o = 0 for     ( I = 1     to     N ) {    if     ( d  [ I ] > D1_th )  N_o = N_o + 1 } P_o = N_o / N

Sequence processor 450 produces two temporal measures of distortion: an average temporal-outlier sequence (TOS) length TL_avg and a maximum temporal-outlier length TL_max. An outlier frame for purposes of TOS length is a frame having distortion greater than temporal outlier distortion threshold D 2 _th. As noted above, sequences of frames having distortion can be much more disruptive than single frames with a high level of distortion, even if the average level of distortion in the sequence of frames is comparatively much lower. Therefore, D 2 _th can be selected to be lower than D 1 _th. The average temporal-outlier sequence (TOS) length TL_avg is determined as follows:

The maximum temporal-outlier sequence length TL_max is then obtained from TL_max=max(T[j]).

Note that the distortion thresholds D 1 _th and D 2 _th above can be either fixed or adaptive. For instance, the distortion thresholds can be made to adapt to the amplitude levels of the sample and test signals or the difference in levels between them.

The statistical and temporal parameters described above can be used individually as indicators of the quality of the test sound frames. These parameters can be used as benchmark-reference objective scores to evaluate new releases of sound transmission products, such as network speech or voice products. Also, the parameters are useful during product design to fine tune the parameters of a product or network under design to obtain a desired level of sound quality.

Further, the statistical and temporal parameters described above can also be combined into a weighted objective score M, where M=f(D_avg, D_var, P_o, TL_avg, TL_max). An example function is M=α*D_avg+β*D_var+γ*P_o+δ*TL_avg+ε*TL_max where α, β, γ, δ and ε are constants. These constants can be derived from a variety of sources including psychophysical models and empirical data. The function ‘f’ can also be non-linear, where α, β, γ, δ and ε vary with D_avg.

M can also be mapped onto a subjective scale, where the mapping is determined based on data from subjective tests. This is similar to the PSQM to objective-MOS mapping described in ITIJ-T P.861 section 10.

The weighted objective score M can be used to evaluate network and transmission circuits and systems involved in sound encoding and transmission, such as coder/decoders and transmission channels. For instance, if a variety of transmission channels exist in a network, then each transmission channel can be evaluated using the present invention to determine its suitability for use as a voice channel. Evaluations can also be performed periodically in the network to obtain a voice quality status check on each transmission channel.

Having described and illustrated the principles of the invention in a preferred embodiment thereof, it should be apparent that the invention can be modified in arrangement and detail without departing from such principles. For example, it will be understood by those of ordinary skill in the art that the present invention can be implemented in a variety of contexts including software for execution on a computer, an embedded application on a processor, or an integrated circuit. We claim all modifications and variations coming within the spirit and scope of the following claims.