Fast speaker recognition scoring using I-vector posteriors and probabilistic linear discriminant analysis

Speaker recognition is employed, or considered for deployment, as a system security tool in many applications and systems. Speaker recognition can include both speaker identification and verification. Speaker identification is determining an identity of an audio sample based on an enrolled population of speakers. Speaker verification uses a voice sample in the form of speech to validate a claimed speaker's identity. In particular, users of a given system or application are verified based on their corresponding voices.

An embodiment of the present invention includes a method of, and apparatus for, performing speaker recognition. The method comprises estimating respective uncertainties of acoustic coverage of at least one speech utterance by a first speaker and at least one speech utterance by a second speaker, the acoustic coverage representing respective sounds used by the first speaker and by the second speaker when speaking. The method also comprises representing the respective uncertainties of acoustic coverage in a manner that allows for efficient memory usage by discarding dependencies between uncertainties of different sounds for the first speaker and for the second speaker. The method further comprises representing the respective uncertainties of acoustic coverage in a manner that allows for efficient computation by representing an inverse of the respective uncertainties of acoustic coverage and then discarding the dependencies between the uncertainties of different sounds for the first speaker and for the second speaker. The method still further comprises computing a score between the at least one speech utterance by the first speaker and the at least one speech utterance by the second speaker in a manner that leverages the respective uncertainties of the acoustic coverage during the comparison, the score being indicative of a likelihood that the first speaker and the second speaker are the same speaker.

Representing the respective uncertainties of acoustic coverage in a manner that allows for efficient computation may include (i) accumulating an inverse of independent uncertainties of acoustic coverage for multiple speech utterances by the first speaker and for multiple speech utterances by the second speaker; (ii) transforming accumulated inverses of the independent uncertainties of acoustic coverage; and (iii) discarding dependencies between the uncertainties of different sounds represented in the transformed accumulated inverses to produce respective diagonalized, transformed accumulated inverses. Computing the score may include using the respective diagonalized, transformed accumulated inverses.

Within the context of the foregoing embodiments, a method of speaker recognition can include receiving, by a computer system, a set of signals corresponding to a set of speech utterances. The method can further include computing, for each speech utterance of the set of speech utterances, a corresponding identity vector (i-vector), a diagonalized approximation of a covariance matrix of the corresponding i-vector, and a diagonalized approximation of an equivalent precision matrix associated with the corresponding i-vector. Such a computation can be represented, for example, by Equations 31 and 34, described below. The method can further include computing a score based on the i-vectors, the diagonalized approximations of covariance matrices, and the diagonalized approximations of equivalent precision matrices computed, the score being indicative of a likelihood that two sets of utterances belong to the same speaker. The score can be computed for each speaker of a number of speakers known to the computer system, in the case of speaker identification, or for a particular speaker, in the case of speaker verification, where the speakers are known to the computer system by way of parameters that had been previously stored in an associated database. Such a computation can be represented, for example, by Equation 39, described below. The method can also include determining the identifier corresponding to the speaker having the highest score.

In another embodiment, computing the i-vector for a speech utterance of the set of speech utterances includes extracting acoustic features from the signal corresponding to the speech utterance and computing the i-vector based on the acoustic features extracted.

In another embodiment, the method includes computing a set of vectors representing projected first order statistics corresponding to the set of speech utterances based on the i-vectors and the diagonalized approximations of the equivalent precision matrices computed. Such a computation can be represented by Equation 35, described below. The method can further include computing a diagonalized approximation of a cumulative equivalent precision matrix for the set of speech utterances based on the diagonalized approximations of precision matrices computed for each i-vector. Such a computation can be represented by Equation 36, described below. The method can further include diagonalizing a transformation of the diagonalized approximation of the cumulative equivalent precision matrix computed. Such a diagonalization can be represented by Equation 37, described below. Computing the score can be based on the set of projected first order statistics, the diagonalized approximation of the cumulative equivalent precision matrix, and the diagonalized transformation of the diagonalized approximation of the cumulative equivalent precision matrix.

The method can further include maintaining, by the computer system, a set of i-vectors for each speaker known to the computer system. The method can additionally include computing, for each i-vector of a set of i-vectors corresponding to a speaker known to the computer system, a diagonalized approximation of a covariance matrix of the i-vector corresponding to the speaker known to the computer system and a diagonalized approximation of an equivalent precision matrix associated with the i-vector corresponding to the speaker known to the computer system. The method can additionally include computing a set of projected first order statistics, for each speaker known to the computer system, based on the set of i-vectors associated with the speaker known to the computer system and the diagonalized approximations of the equivalent precision matrices computed. Such a computation can be represented by Equation 35, described below, for example. The method can further include computing a diagonalized approximation of a cumulative equivalent precision matrix for each speaker known to the computer system based on the diagonalized approximations of precision matrices computed for each i-vector in the set of i-vectors corresponding to the speaker known to the computer system. Such a computation can be represented by Equation 36, described below, for example. The method can further include diagonalizing a transformation of the diagonalized approximation of the cumulative equivalent precision matrix computed for each speaker known to computer system. Such a diagonalization can be represented by Equation 37, described below, for example. Computing the score can include computing the score based on the set of projected first order statistics, the diagonalized approximation of the cumulative equivalent precision matrix, and the diagonalized transformation of the diagonalized approximation of the cumulative equivalent precision matrix associated with the speaker known to the computer system. The method can further include storing the diagonalized approximation of a covariance matrix of the i-vector in the database.

In another embodiment, determining if the set of speech utterances corresponds to one or any of the number of speakers known to the computer system includes comparing the score computed to a threshold.

In another embodiment, the computer system includes one or more processors and or one or more computer devices.

An apparatus can include a processor, and a memory, with computer code instructions stored thereon. The processor and the memory, with the computer code instructions stored thereon, are configured to cause the apparatus to receive a set of signals corresponding to a set of speech utterances. The processor and the memory are further configured to compute, for each speech utterance of the set of speech utterances, a corresponding identity vector (i-vector), a diagonalized approximation of its covariance matrix, and a diagonalized approximation of an equivalent precision matrix associated with the corresponding i-vector. Such a computation can be represented, for example, by Equations 31 and 34, described below. The processor and the memory are further configured to compute a score based on the i-vectors, the diagonalized approximations of their covariance matrices, and the diagonalized approximations of equivalent precision matrices computed, the score being indicative of a likelihood that two sets of utterances, represented by the corresponding i-vectors and diagonalized covariances, belong to the same speaker. Computing the score can be done for each speaker of a number of speakers known to the computer system, in the case of speaker identification, or for a particular speaker, in the case of speaker verification. Such a computation can be represented, for example, by Equation 39, described below. The processor and the memory are further configured to determine if the set of speech utterances corresponds to any of the number of speakers known to the computer system based on the scores computed.

In an embodiment, a computer-readable medium has computer code stored thereon, and the computer code, when executed by a processor, is configured to cause an apparatus to operate in accordance with the foregoing embodiments.

The foregoing will be apparent from the following more particular description of example embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments of the present invention.

FIG. 1 is a block diagram of a speaker-verification server employing voice biometrics in user authentication according to at least one example embodiment.

FIG. 2 is a block diagram illustrating functional components of the speaker recognition server, according to at least one example embodiment.

FIG. 3 is a flow diagram that illustrates an example embodiment of the present invention.

FIG. 4 is a flow diagram illustrating an example embodiment of the present invention executed by a processor.

A description of example embodiments of the invention follows.

FIG. 1 is a block diagram of a speaker-recognition server 100 employing voice biometrics in user authentication according to at least one example embodiment. At least one speech signal 10 is provided by a user to the speaker recognition server 100 . The at least one speech signal 10 , a sample of the user's voice, may be one or more utterances spoken by the user. When performing speaker verification, the speaker recognition server 100 is configured to validate a claimed identity 115 of the user based on the received speech signal(s) 10 and model parameters for that speaker, including one or more i-vectors and their corresponding diagonalized covariances, extracted from the user's utterances of the speaker recognition server 100 , collected, for example, during an enrollment phase and stored in a database, or data storage device, 120 . In particular, the speaker recognition server 100 is configured to compare voice characteristics extracted from the speech signal(s) 10 , provided by the user to be authenticated, against voice characteristics corresponding to the claimed identity (ID) 115 of that user as retrieved from the database or, for that matter, of all users known to the speaker recognition server 100 , if the claimed ID 115 is not known and the system must perform speaker identification. FIG. 1 shows the block diagram of a speaker verification system when the claimed ID 115 is provided; however, the speaker recognition server 100 can be configured also to identify a speaker from received speech signal(s) based on a comparison to a plurality of speakers known to the system when the claimed ID is not provided.

According to at least one example embodiment, the speaker recognition server 100 includes an i-vector based speaker recognition module 110 , configured to perform i-vector based speaker recognition. I-vector modeling has become the standard approach for speaker recognition due to its high accuracy and small speaker model footprint. The speaker recognition server 100 outputs a decision 50 with regard to the user verification or identification. For user (i.e., speaker) verification, the decision 50 is indicative of whether the voice characteristics extracted from speech signal 10 sufficiently match (e.g., match above a particular threshold) the voice characteristics represented by the speaker model (voiceprint) stored in 120 for the claimed ID.

According to at least one aspect, the decision 50 is provided to one or more modules to act upon it. For example, the verification decision 50 may be provided to an intelligent voice recognition module, which is configured to allow user access to a requested service or request more information or data from the user based on the speaker verification decision 50 . The speaker verification decision may be used by a speech recognition system to determine user-specific speech recognition parameters to be employed.

In addition to speaker verification as described above, the decision can represent the identify of a speaker within an enrolled population of speakers in the case of speaker identification. For user identification, the decision 50 is indicative of which speaker known to the system best matches the input speech signal(s). This is known as closed-set speaker identification. In the event that none of the speakers known to the system have a sufficient match to the input speech signal(s), the system may output a value indicating this, i.e., “none-of-the-above”. In the context of identification, the inclusion of the output indicating an insufficient match to any speaker known to the system is known as open-set speaker identification. As with speaker verification, the decision 50 may be provided to one or more modules to act upon it. For example, a public security system may use the decision to notify an agent that the input speech signal(s) appear to match a specific criminal within a database of known perpetrators.

Some existing speaker recognition systems employ a probabilistic linear discriminant analysis (PLDA) model, which exploits intrinsic i-vector uncertainty, and is known to provide good accuracy for short speaker segments. In particular, an approach, referred to as full posterior PLDA (FP-PLDA), is employed in performing i-vector based speaker recognition. However, the FP-PLDA is computationally much more expensive than the standard i-vector based PLDA approach. According to at least one example embodiment, a diagonalized full posterior PLDA (DFP-PLDA) method is employed by the speaker recognition server 100 and, specifically, by the procedure included in the i-Vector Based Speaker Recognition Module 110 and the additional parameters stored in the Data Storage 120 . According to at least one aspect, the DFP-PLDA method substantially reduces the computational costs of the FP-PLDA approach while providing similar, or almost similar, accuracy as the FP-PLDA approach.

FIG. 2 is a block diagram illustrating functional components of the speaker recognition server 100 , according to at least one example embodiment. In particular, the i-vector based speaker verification module 110 includes a front-end system 112 , an i-vector extraction module 114 , a DFP-PLDA module 115 , and a decision module 118 . The front-end system 112 is configured to extract feature coefficients from a received speech signal 10 . Examples of voice features include Mel frequency cepstral coefficients (MFCCs), linear prediction cepstral coefficients (LPCC), perceptual linear predictive (PLP) cepstral coefficients, or the like. The i-vector extraction module 114 is configured to forward an extracted i-vector 34 and a diagonal approximation of the covariance matrix 36 to the DFP-PLDA module 115 . The DFP-PLDA Module 115 can then store the i-vector 38 and diagonal approximation of the covariance matrix 42 in data storage 120 . In performing speaker identification, the DFP-PLDA Module 115 produces multiple scores 44 . In performing speaker verification, the DFP-PLDA Module 115 produces a single score 44 .

FIG. 3 is a flow diagram that illustrates an embodiment of the present invention. The flow diagram includes a method of, and corresponding apparatus for, performing speaker recognition. The method comprises estimating respective uncertainties of acoustic coverage of at least one speech utterance by a first speaker and at least one speech utterance by a second speaker, the acoustic coverage representing respective sounds used by the first speaker and by the second speaker when speaking ( 302 ). By estimating the respective uncertainties, a full covariance is obtained, and this captures uncertainty in i-vector modeling due to insufficient acoustic coverage from short utterances. Covariance provides for an uncertainty measurement that may be used to know where, in a phonetic space, there are areas of high, little, or no coverage. For example, in an enrollment speech database, a speaker or many speakers have uttered speech of “1, 2, 3, 4,” and an incoming speech signal represents a speaker's uttered speech of “5, 6, 7, 8.” In this example, one should understand that there are areas in the phonetic space that have little or no coverage, making identification and verification more uncertain than if the speaker(s) had uttered the same phonemes as in the enrollment speech “5, 6, 7, 8.” Thus, use of covariance can be useful, but it comes at a cost of higher memory usage and higher computations. Discarding dependencies between uncertainties of different sounds in a computationally appropriate way can be useful in addressing both costs while simultaneously increasing accuracy, compared to non-covariance techniques, during identification and verification processes.

Continuing to refer to FIG. 3 , the method also comprises representing the respective uncertainties of acoustic coverage in a manner that allows for efficient memory usage by discarding dependencies between uncertainties of different sounds within the respective acoustic coverage for the first speaker and for the second speaker ( 304 ). When viewing the full covariance as containing information regarding the uncertainty, discarding dependencies between uncertainties of different sounds uses a diagonal covariance in which non-diagonal terms corresponding to correlations between different dimensions are removed. The method also comprises representing the respective uncertainties of acoustic coverage in a manner that allows for efficient computation by representing an inverse of the respective uncertainties of acoustic coverage and then discarding the dependencies between the uncertainties of different sounds within the respective acoustic coverage for the first speaker and for the second speaker ( 306 ). The inverse of the respective uncertainties with discarded dependencies between the uncertainties refers to a precision matrix. The method also includes computing a score between the at least one speech utterance by the first speaker and the at least one speech utterance by the second speaker that leverages the respective uncertainties of the acoustic coverage during the comparison, the score being indicative of a likelihood that the first speaker and the second speaker are the same speaker ( 308 ). The method or corresponding apparatus can be used to perform speaker identification and can also be used to perform speaker verification.

It should be understood that the term “efficient” in the context of efficient memory usage and efficient computation can be any amount of efficiency that is an improvement over techniques that do not discard dependencies between uncertainties of different sounds for the speakers. Efficiency improvements can be in the form of percent, such as 5%, 10%, or 50% improvements, or can be in the form of memory size metrics (e.g., KBytes) for the efficient memory usage or number of calculations for the efficient computation. Table III below provides examples of efficiencies corresponding to the DFP-PLDA embodiments disclosed herein relative to FP-PLDA and PLDA systems of the prior art. Table II below provides additional examples of efficiencies in the form of algebraic expressions.

A particular embodiment of the foregoing method may also include representing the respective uncertainties of acoustic coverage in a manner that allows for efficient computation by accumulating a respective inverse of independent uncertainties of acoustic coverage for multiple speech utterances for the first speaker and multiple speech utterances for the second speaker. The accumulating results in cumulative diagonal precision matrices, where a diagonal matrix is implied by use of the term “independent.” The particular embodiment also includes transforming a respective accumulated inverse of the independent uncertainties of acoustic coverage. The particular embodiment may also include discarding dependencies between the uncertainties of different sounds represented in the respective transformed accumulated inverse to produce a respective diagonalized, transformed accumulated inverse. The particular embodiment also includes computing the score by using the respective diagonalized, transformed accumulated inverses.

With respect to the foregoing, the covariance basically captures the uncertainty of coverage in the i-vector modeling due to insufficient acoustic coverage in the case of short utterances. The uncertainty is captured in the form of the variance for the different iVector dimensions where the larger the variance, the higher the uncertainty.

The precision matrix is related to the inverse of the covariance matrix. So say, for simplicity, that the i-vector elements are uncorrelated and the precision matrix's corresponding covariance matrix is diagonal with terms on the diagonal as sigma_ 1 , sigma_ 2 , . . . , sigma_M. The large sigmas from the covariance matrix (indicating high uncertainty) upon inversion have correspondingly small sigmas in the precision matrix, whereas the small sigmas in the covariance matrix have correspondingly high values (i.e., indicating certainty) in the form of a precision matrix.

In some sense, large diagonal values in the covariance matrix is an indication of which i-vector elements are the most uncertain, and large diagonal values in the precision matrix indicates which precision matrix elements are the most certain.

FIG. 4 is a flow diagram 400 illustrating an example embodiment of a process employed by the present invention. First, the process receives a set of signals or representations of signals that correspond to speech utterances at a computer system ( 402 ). Then, the process computes, for each utterance: a corresponding identity vector (i-vector), a diagonalized approximation of a covariance matrix of the corresponding i-vector, and a diagonalized approximation of an equivalent precision matrix associated with the corresponding i-vector ( 404 ). Then, the process computes a score, for each speaker of a number of speakers known to the computer system, based on the i-vectors, the diagonalized approximations of covariance matrices, the diagonalized approximations of equivalent precision matrices, and, optionally, the diagonalized transformation of the diagonalized approximation of a cumulative equivalent precision matrix ( 406 ). In the case of verification, the process computes a score, for the speaker model parameters corresponding to the claimed ID and speech, to verify the speech is from the speaker corresponding to the claimed ID based on the i-vectors, the diagonalized approximations of covariance matrices, the diagonalized approximations of equivalent precision matrices, and the diagonalized transformation of the diagonalized approximation of the cumulative equivalent precision matrix ( 406 ). The score is indicative of a likelihood of a correspondence between the set of utterances received and the speaker ( 406 ). Then, the process, for identification, determines if the speech utterances correspond to any of the number of speakers known to the computer system based on the scores computed, and for verification, determines whether the speaker corresponding to the claimed ID matches the speech ( 408 ).

I-Vector Model

The i-vector extraction module 114 is configured to generate an i-vector based on the extracted feature coefficients. Given a sequence of feature vectors, referred to as χ={x 1 , x 2 , . . . , x τ }, extracted from the speech signal 10 for an individual user, one i-vector is generated by the i-vector extraction module 114 based on an a-priori distribution of the i-vectors and the i-vector model described as

s=u+Tw   (1)

where u is a Universal Background Model (UBM) super-vector representing statistical parameters, i.e., a plurality of mean vectors, associated with distributions of feature vectors extracted from training background audio data, s is a super-vector representing corresponding statistical parameters, i.e., a plurality of mean vectors, associated with distributions of feature vectors corresponding to the individual speaker, T is a low-rank rectangular matrix including vectors which span a subspace representing potential variations of the super-vector s with respect to the background statistical parameters in u, and w is an i-vector corresponding to the individual speaker characteristics in the spoken utterance.

According to at least one aspect, w is a realization of a latent variable W, of size M, having a standard normal prior distribution. Given T, and the set of feature vectors χ={x 1 , x 2 , . . . , x τ } extracted from the speech segment 10 , it is possible to compute the likelihood of χ given the model (1), and a value for the latent variable W. The i-vector w, which represents the speech segment, is computed as the Maximum a Posteriori (MAP) point estimate of the variable W, i.e., the mean μ χ of the posterior distribution P W|χ (w). Assuming a standard normal prior for W, the posterior probability of W given the acoustic feature vectors χ is Gaussian:

W |χ˜ (μ χ ,Γ χ −1 )  (2)

with mean vector and precision matrix:

μ χ =Γ χ −1 T*Σ −1 f χ   (3a)

Γχ= I+Σ c=1 C N χ (c) T (c) *Σ (c)−1 T (c) ,  (3b)

respectively. In these equations, N χ (c) are the zero-order statistics estimated on the c-th Gaussian component of the UBM for the set of feature vectors χ, f χ is the super-vector stacking the first-order statistics f χ (c) , centered around the corresponding UBM means:

f χ ( c ) = ∑ t ⁢ ( γ t ( c ) ⁢ x t ) - N χ ( c ) ⁢ u ( c ) ( 3 ⁢ c )

Σ (c) is the UBM c-th covariance matrix, Σ is a block diagonal matrix having the matrices Σ (c) as its entries, T (c) is the sub-matrix of T corresponding to the c-th mixture component, and γ t (c) is the c-th occupation probability of a feature vector x t of χ.

Gaussian Full Posterior Distribution PLDA Model

An utterance z is represented in the standard Gaussian PLDA model by the i-vector posterior mean μ, which is assumed to be the combination of three terms:

μ= m+Uy+e,   (4)

where m is the i-vector mean, y is a speaker factor that has a normal prior distribution, matrix U typically constrains the speaker factor to be of lower dimension than the i-vectors, and the residual noise prior is Gaussian with full precision matrix Λ. That is:

Y ˜ (0, I ),Σ˜ (0,Λ −1 ).  (5)

Considering the uncertainty associated with the extraction process of the i-vector, which is represented by its posterior covariance, the PLDA model in (5) is extended to exploit this additional information. This extended model, referred to as the PLDA based on the full posterior distribution of W given χ, assumes that the feature vectors x i of an utterance z i are mapped to an i-vector μ according to the probability distribution P wi |χi (μ). The extended model, or full-posterior PLDA (FP-PLDA) model, may be described as:

μ= m+Uy+ē,   (6)

where the difference with equation (4) is that the distribution Ē of the residual noise ē in equation (6) is utterance-dependent. The i-vector associated with the utterance z i is again the mean μ i of the i-vector posterior W i|χi , but the priors of the PLDA parameters are given by:

Ē i ˜N(0,Λ −1 +Γ i −1 )˜ N (0,Λ eq,i −1 ), Y˜N (0, I ),  (7)

where Γ i is the precision matrix produced by the i-vector extractor, and the equivalent precision matrix Λ eq,i is

Λ eq,i =(Λ −1 +Γ i −1 ) −1 .  (8)

According to the FP-PLDA model, the likelihood that a set of n i-vectors μ 1 . . . μ n , or a corresponding set of utterances z 1 . . . z n , belongs to the same speaker may be computed as:

log ⁢ ⁢ P ⁡ ( μ 1 ⁢ ⁢ … ⁢ ⁢ μ n ❘ H s ) = ∑ i ⁢ [ 1 2 ⁢ log ⁢  Λ eq , i  - M 2 ⁢ log ⁢ ⁢ 2 ⁢ ⁢ π - 1 2 ⁢ ( μ i - m ) T ⁢ Λ eq , i ⁡ ( μ i - m ) ] - 1 2 ⁢ log ⁢  Λ y  + 1 2 ⁢ μ y T ⁢ Λ y ⁢ μ y - S 2 ⁢ log ⁢ ⁢ 2 ⁢ ⁢ π , ( 9 )

where M is the i-vector dimension, S is the speaker factor dimension, and the parameters Λ y , μ y are defined as:

Λ y =I+Σ i U T Λ eq,i U   (10a)

μ y =Λ y −1 U T Σ i Λ eq,i (μ i −m )  (10b)

The equations (10) are similar to their equivalents in the PLDA model except that in the PLDA model Λ replaces Λ eq,i , which accounts for the utterance-dependent i-vector precision matrix in the FP-PLDA model.

Complexity Analysis

Given a set of n enrollment utterances u e 1 . . . u e n for a target speaker, and a set of m test utterances of a single, unknown, speaker u t 1 . . . u t m , the speaker verification log-likelihood ratio s is:

s = log ⁢ l ⁡ ( u e 1 ⁢ … ⁢ ⁢ u e n , u t 1 ⁢ … ⁢ ⁢ u t m | H s ) l ⁡ ( u e 1 ⁢ … ⁢ ⁢ u e n | H s ) ⁢ l ⁡ ( u t 1 ⁢ … ⁢ ⁢ u t m | H s ) , ( 11 )

where H s is the hypothesis that the two sets of utterances belong to the same speaker. The naive implementations of classical PLDA and of FP-PLDA have similar computational complexity. However, a common application scenario consists of a speaker detection task where a set of utterances of a single test speaker has to be verified against the utterances of a set of predefined target speakers. In this scenario, a smart implementation of PLDA allows some of the terms required for the evaluation of the speaker verification log-likelihood ratio to be pre-computed, thus the per-trial scoring complexity is greatly reduced. The FP-PLDA model does not allow the pre-computation of most of the terms of the scoring function, due to the presence of the utterance-dependent i-vector precision matrix in (9), thus its complexity may not be reduced.

A. Log-Likelihood Computation

The complexity of the log-likelihood computation accounts for three separate contributions:

per-target costs: operations that can be independently performed on target sets (referred to as per-target costs),

per-test costs: operations that can be independently performed on test sets (referred to as per-test costs),

per-trial costs: operations that jointly involve both the target and the test sets (referred to as per-trial costs),

These distinctions are not relevant for naïve scoring implementations, but are relevant, instead, in the “predefined target speakers scenario” because the per-target terms can be pre-computed, and per-test terms need to be computed only once regardless of the number of target speakers.

Since the posteriors of the speaker variable y are computed on different sets, the parameters of the posterior distributions of y (10) to a generic set G are conditioned as:

Λ y|G =I+Σ iεG U T Λ eq,i U   (12a)

μ y|G =Λ y −1 U T Σ iεG Λ eq,i (μ i −m )  (12b)

The indexes of the sum in this equation, and in the following equations, are to be interpreted as running over all the utterances of the set. Replacing (9) in (11), the speaker verification log-likelihood ratio for a target set E and a test set T can be written as:

where the scoring function σ is defined as:

σ( G )=−½ log|Λ y|(G) |+½μ y|(G) T Λ y|(G) μ y|(G)   (14)

Analysis is restricted to the term σ(E,T) because the computation of the log-likelihood ratio is dominated by the term σ(E, T).

B. Complexity of the Standard Gaussian PLDA

As described above, standard PLDA corresponds to a FP-PLDA with Γ i −1 =0 for all i-vectors. Thus, Λ eq,i =Λ for all i-vectors, and the speaker variable posterior parameters become:

where n E and n T are the number of target and test segments, respectively, and F E and F T are the projected first order statistics defined as:

F E =M Σ iεE (μ i −m ),

F T =M Σ iεT (μ i −m )  (16)

and M=U T Λ is a S×M matrix, where S is the PLDA speaker sub-space dimension. Using these definitions, the scoring function σ(E, T) can be rewritten as:

σ( E,T )=−½ log|Λ y|(E,T) |+F E T Λ y|(E,T) −1 F T +½ F T T Λ y|(E,T) −1 F T +½ F E T Λ y|(E,T) −1 F E   (17)

Computing the projected statistics (16) has complexity O(NM)+O(MS), where N is the number of utterances in the set. The F E and F T statistics are per-set computations because they are computed for the target and test sets independently. Their complexity is O(NM) because N i-vectors of dimension M are summed.

For the naïve scoring implementation, the computation of the score function σ(E, T) given the F G statistics, requires computing Λ y|(E,T) −1 and its log-determinant. These computations have complexity O(S 3 ) because, for standard PLDA, the term U T ΛU can be pre-computed. Given Λ y|(E,T) −1 , scoring σ(E, T) has complexity O(S 2 ). The same considerations apply to the less expensive computation of σ(E) and σ(T). Thus, the overall per-trial complexity is O(S 3 ).

For the speaker detection with known target sets, in the naïve implementation, the computation and inversion of Λ y|(E,T) dominates the scoring costs. However, in standard PLDA, this factor depends only on the number (n T +n E ) of the target and test utterances (15). Since each set of target utterances E k and the number of test utterances n T are known, it is possible to pre-compute the corresponding Λ y|(E k , T) −1 , and its log-determinant. Moreover, since the statistics F E k are also known in advance, the terms of the scoring function ½F E k T Λ y|(E k ,T) −1 can be pre-computed. These terms are small S-sized vectors. Since the term depending only on the test statistics F T must be evaluated just once for the whole set of K targets, its computation has a per-test, rather than a per-trial, cost. Every function σ(E k , T) can be computed in O(S), and each term σ(E k ) can be pre-computed. Given the statistics, the term σ(T) has a per-set complexity of O(S 2 ). The overall per-set cost, including statistics computations, is then O(NM)+O(MS), whereas the per-trial cost is O(S).

C. Full-Posterior PLDA

The main difference between the standard PLDA and the FP-PLDA approach is that in PLDA Λ y|(E,T) depends only on the number of i-vectors in the set (15), whereas in FP-PLDA, it also depends on the covariance of each i-vector (12) in the test set T. This does not allow applying to FP-PLDA the optimizations illustrated in the previous section.

The speaker variable posterior parameters can still be written as:

Λ y|(E,T) =I +(Λ eq,E +Λ eq,T )  (18a)

μ y|(E,T) =Λ y −1 ( F eq,E +F eq,T )  (18b)

where

F eq,G =U T Σ iεG Λ eq,i (μ i −m )

Λ eq,G =U T (Σ iεG Λ eq,i ) U   (18c)

and the scoring function σ(E, T) is:

σ( E,T )=−½ log|Λ y|(E,T) |+½ F eq,E T Λ |(E,T) −1 F eq,E +½ F eq,T T Λ y|(E,T) −1 F eq,T +F eq,E T Λ y|(E,T) −1 F eq,T   (19)

Computing the posterior parameters (18) has a complexity O(NM 3 )+O(M 2 S), mainly due to the computation of Λ eq,i and is much higher than the O(NM)+O(MS) complexity of standard PLDA approach. However, these computations are required only for a new target or test speaker. These per-set costs are comparable to the costs O(NM 3 ) of the i-vector extraction. Given the statistics, Λ y|(E,T) can be computed with complexity O(S 2 ) and its inversion complexity is O(S 3 ). The computation of the remaining terms requires O(S 2 ); thus, the overall per-trial complexity is O(S 3 ). Since the posterior parameter Λ y|(E,T) cannot be pre-computed as in standard PLDA, the per-trial complexity is the same also for the fixed set of target speaker scenarios. Table I compares the log-likelihood computation complexity of the naïve and optimized PLDA implementations, with respect to the complexity of FP-PLDA. The computational requirements of the FP-PLDA system are much more expensive than standard PLDA due to the computation of Λ y|(E,T) that increase both the per-set and the per-test costs. Table I, above also shows that the per-trial costs of FP-PLDA is two orders of magnitude larger than the optimized implementation of PLDA.

Approximated Full-Posterior PLDA

The proposed FP-PLDA model allows improving speaker recognition performance. However, as shown before, its per-trial score complexity greatly increases compared to the standard PLDA approach. Moreover, additional memory is required not only to store the covariance of the i-vector, but also to store some pre-computed matrices for computational efficiency. Embodiments of the present invention address these issues by applying a sequence of diagonalization operators that approximates the full matrices needed for i-vector scoring. Three approximations for fast scoring can be created with a very small impact on the FP-PLDA system accuracy.

A. Diagonalized i-Vector Posterior

The first, straightforward, approximation includes approximating the i-vector posterior covariance by a diagonal matrix:

Γ i −1 ←Γ i −1 ∘I   (20)

where ∘ is the element-wise product operator. Using a diagonal i-vector posterior covariance allows significant memory savings for storing the target models (O(M) rather than M 2 ). However, even though the i-vector posterior covariance is diagonal, the matrices Λ eq,i of (8) remain full. Thus, this approach alone does not give any computational advantage with respect to the standard FP-PLDA. Moreover, this approximation is related to the i-vector extractor rather than to the PLDA classification model.

B. Diagonalized Residual Covariance

In order to speed-up the computation of the covariance of the residual term Ē i (7), it is diagonalized, which not only reduces the scoring complexity, but also allows the exact PLDA solution to be recovered for long enough utterances. In particular, the precision matrix Λ of the PLDA residual term E can be eigen-decomposed as:

Λ= V Λ D Λ V Λ T

where V Λ is an orthogonal matrix, and D Λ is a diagonal matrix. The precision matrix of Ē i thus can written as:

Λ eq,i =(Λ −1 +Γ i −1 ) −1 =( V Λ D Λ −1 V Λ T +Γ i −1 ) −1 =V Λ ( D Λ −1 +V Λ T Γ i −1 V Λ ) −1 V Λ T   (21)

The proposed approximation consists in replacing the term V Λ T Γ i −1 V Λ by a diagonal matrix V Λ T Γ i −1 V Λ ∘ I.

In order to analyze the complexity of the scoring with this approximation, Λ eq,i D is defined as:

Λ eq,i D =( D Λ −1 +V Λ T Γ i −1 V Λ ∘I ) −1   (22)

and the approximated Λ eq,i (21) is rewritten as:

Λ eq,i =V Λ Λ D eq,i V ζ T   (23)

The statistics F eq,E and F eq,T can be computed by replacing (23) in (18). The approximated speaker identity posterior covariance can be rewritten as:

Λ y|(E,T) =I+U T V Λ (Λ eq,E D +Λ eq,T D ) V Λ T U   (24)

where

Λ eq,E D =Σ iεE Λ eq,i D ,Λ eq,T =Σ iεT Λ eq,i D   (25)

Thus, Λ y|(E,T) depends on the i-vectors covariance only through the diagonal statistics Λ eq,E D and Λ eq,T D

C. Diagonalized Speaker Identity Posterior

A third approximation, which further decreases the scoring complexity, includes a joint approximated diagonalization of the speaker identity posteriors. Such joint diagonalization does not introduce approximations in standard PLDA. The term U T ΛU in (15) is decomposed as:

U T ΛU=V Y D Y V Y T   (26)

where V Y is an orthogonal matrix and D Y is diagonal. The speaker identity posterior covariance is then given by:

Λ y|(E,T) −1 =( I +( n E +n T ) V Y D Y V Y T ) −1 =V Y ( I +( n E +n T ) D Y ) −1 V Y T   (27)

where the factor I+(n E +n T )D Y is diagonal.

The same decomposition of U T ΛU can be applied to FP-PLDA obtaining:

Λ y|(E,T) −1 =( I+V Y ( {circumflex over (D)} eq,E +{circumflex over (D)} eq,T) V Y T ) −1 =V Y ( I +( {circumflex over (D)} eq,E +{circumflex over (D)} eq,T )) −1 V Y T    (28)

where

{circumflex over (D)} eq,E =V Y T U T Λ eq,E UV Y   (29a)

{circumflex over (D)} eq,T =V Y T U T Λ eq,T UV Y   (29b)

In contrast with standard PLDA, the matrices {circumflex over (D)} eq,E and {circumflex over (D)} eq,T are not diagonal. The proposed diagonalization includes replacing these terms by the diagonal matrices:

D eq,E ={circumflex over (D)} eq,E ∘I,D eq,T ={circumflex over (D)} eq,T ∘I   (30)

The impact of this approximation becomes irrelevant with the increase of the utterance duration, as it happens with the diagonalized residual covariance approach.

D. Diagonalized FP-PLDA

The three diagonalization approaches, illustrated in the previous sub-sections, can be efficiently combined to speed-up the computation of the FP-PLDA log-likelihood ratios. Embodiments of present invention provide a sequence of operations for computing the scoring function σ(E, T) of a fully diagonalized FP-PLDA. Table II compares the complexity of the different approaches.

In every presented approximation, replacing the diagonalizing operator ∘ I by the operator ∘ 1, where 1 is a matrix of ones, produces the standard FP-PLDA solution. For each diagonalization approach, a matrix operator Q is defined such that Q=I when the diagonalization is applied, and Q=1 otherwise. Q Γ , Q Λ , Q Y are defined as the operators associated to i-vector covariance diagonalization, residual covariance diagonalization, and speaker identity posterior covariance diagonalization, respectively. For each utterance, the i-vector covariance matrix approximation is defined as:

(Γ i D ) −1 =Γ i −1 ∘Q Γ or (Γ i D ) −1 =(Γ i ∘Q Γ ) −1   (31)

An S×M matrix W is also defined as:

W=V Y T U T V Λ   (32)

The operations for computing the scoring function σ(E, T) are:

1) For each utterance compute:

Γ Λ,i −1 =V Λ T (Γ i D ) −1 V Λ ∘Q Λ   (33)

and the approximated equivalent precision matrix:

Λ eq,i D =( D Λ −1 +Γ Λ,i −1 ) −1   (34)

2) For each set G compute:

F eq,G =W Σ iεG Λ eq,i D V Λ T (μ i −m )  (35)

and the diagonalized approximation of a cumulative equivalent precision matrix:

Λ eq,G D =Σ iεG Λ eq,i D   (36)

and its diagonalized transformation:

D eq,G =WΛ eq,G D W T ∘Q Y   (37)

3) For each trial, compute:

(Λ y|(E,T) D ) −1 =( I+D eq,E +D eq,T )) −1   (38)

σ( E,T )=−½ log|(Λ y(E,T) D ) −1 |+½ F eq,E T (Λ y|(E,T) D ) −1 F eq,E +½ F eq,T T (Λ y|(E,T) D ) −1 F eq,T +F eq,E T (Λ y|(E,T) D ) −1 F eq,T   (39)

Equation (31) can be considered part of the i-vector extractor. Equation (31) also directly impacts the complexity of the extractor. In fact, if Q Γ =1 the full covariance of the i-vector has to be computed with complexity O(NM 3 ). On the other hand, if Q Γ =I, only the diagonal of the i-vector posterior is needed.

Table II summarizes the complexity of the approaches for different settings of the diagonalizing operators Q Γ , Q Λ , Q Y . Combining different approximations notably reduces the complexity with respect to the individual contribution of each diagonalization. Applying the sequence of the proposed approaches reduces both the per-set and per-trial scoring computations, thus shrinking the computational gap between standard PLDA and FP-PLDA.

Table III provides an example of the results obtained on a liveness detection task, in terms of percent Equal Error Rate, minimum Decision Cost Function (DCF08), model size (in KB), scoring time, and total time (i-vector extraction+scoring) in seconds. In these experiments, every utterance was processed after Voice Activity Detection, extracting every 10 ms, 19 Mel frequency cepstral coefficients, and the frame log-energy on a 25 ms sliding Hamming window. This 20-dimensional feature vector was subjected to short time mean and variance normalization using a 3 second (s) sliding window, and a 45-dimensional feature vector was obtained by stacking 18 cepstral (c 1 -c 18 ), 19 delta (Δc 0 -Δc 18 ) and 8 double-delta (ΔΔc 0 -ΔΔc 7 ) parameters.

A gender-independent i-vector extractor was trained. The training was based on a 1024-component diagonal covariance gender-independent UBM, and on a gender-independent T matrix, trained using NIST SRE 2004-2010 and additionally the Switchboard II, Phases 2 and 3, and Switchboard Cellular, Parts 1 and 2 datasets for a total of 66140 utterances. The i-vector dimension was fixed to d=400.

PLDA models were trained with full-rank channel factors, using 200 dimensions for the speaker factors, using the NIST SRE 2004-2010 datasets, for a total of 48568 utterances of 3271 speakers.

The i-vectors of the PLDA models were whitened and normalized according to the Projected Length Normalization:

The results in the first and second rows of Table III clearly show how valuable is the “uncertainty” information exploited by the FP-PLDA approach. FP-PLDA is able to reduce the percent EER and the cost function by more than 40%, but it introduces a huge increase of memory and computational costs. The diagonalized FP-PLDA approach dramatically reduces these costs, still improving the PLDA performance by approximately 35%.

The Diagonalized FP-PLDA model exploits the uncertainty of the i-vector extraction process. By applying an appropriate sequence of diagonalization operators that approximate the full matrices needed for i-vector scoring, a computational complexity in scoring is obtained that is comparable to PLDA, but with a performance that remains comparable to the more accurate FP-PLDA models. Other advantages of this approach are the reduced memory costs with respect to FP-PLDA, and the possibility of applying the same technique in combination with the Factorized Subspace Estimation approach.

Detailed Complexity Analysis

A. Standard FP-PLDA

Most of the operations detailed in the method are redundant for standard FP-PLDA. However, the resulting asymptotic complexity for FP-PLDA is the same, and the operations serve as a reference for describing the contribution of the different approximations on the overall scoring complexity.

Equation (33) has a complexity O(NM 3 ).

Since Γ Λ,i −1 is a full matrix, Equation (34) has a complexity O(NM 3 ), and produces a full Λ eq,i D matrix.

The computation of the statistics in (35) and (36) have an overall complexity O(NM 2 )+O(MS), and Λ eq,G D is again a full matrix.

The computation of D eq,G in (37) has a complexity of O(M 2 S) and, again, results in a non-diagonal matrix.

The per-trial term in equation (38) has a complexity O(S 3 ).

Finally, equation (39) can be computed in O(S 2 ).

Combining all these operations gives an overall O(NM 3 )+O(M 2 S) per-set complexity, and O(S 3 ) per-trial complexity.

B. Diagonalized I-Vector Covariance

Diagonalization of the i-vector posterior covariance corresponds to setting Q Γ =I.

Although (Γ i D ) −1 is diagonal, equation (33) still requires O(NM 3 ) operations.

Since Γ Λ,i −1 is full, all the remaining operations have the same complexity of the standard FP-PLDA.

The overall complexity is, therefore, the one given in the previous sub-section.

C. Diagonalized Residual Covariance

The complexity of the diagonalized residual covariance approximation is related to the use of the diagonalized i-vector covariance approximation.

In particular:

Equation (33) has complexity O(NM 3 ). However, if (Γ i D ) −1 is diagonal, Γ Λ,i −1 can be evaluated in O(NM 2 ) operations because only the diagonal of the right hand side of the equation is needed.

Since Γ Λ,i −1 is diagonal, Equation (34) has a complexity O(NM).

The computations of the statistics in Equation (35) requires O(NM 2 )+O(MS) operations.

The terms in Equation (36) can be computed in O(NM) operations.

Equation (37) has a per-set complexity O(MS 2 ).

Equation (38) has a per-trial complexity O(S 3 ).

Finally, Equation (39) can be computed in O(S 2 ).

Overall, the per-set complexity is O(NM 3 )+O(MS 2 ) and the per-trial complexity is O(S 3 ). However, if this approximation is preceded with the diagonalization of the i-vector posterior covariance, the per-set complexity decreases to O(NM 2 )+(MS 2 ).

D. Diagonalized Speaker Identity Posterior

Again, the complexity of this approximation depends on the sequential application of the first two diagonalization. In particular:

The complexity of equations (33) to (36) depends only on the previous approximations, and is not affected by the diagonalization of the speaker posterior covariance.

Equation (37) has complexity O(M 2 S). However, it can be computed in O(MS) if Λ eq,G D is diagonal, because only the diagonal of the right hand side of the equation is needed.

Since D eq,G is diagonal equation, Equation (38) has a per-trial complexity O(S).

Finally, Equation (39) can be computed in O(S 2 ).

This approximation allows the per-trial complexity to be reduced from O(S 3 ) of standard FP-PLDA to O(S 2 ). The per-set complexity is also heavily dependent on the use of the previous approximations.

Embodiments or aspects of the present invention may be implemented in the form of hardware, software, or firmware. If implemented in software, the software may be any form of software capable of performing operations consistent with the example embodiments disclosed herein. The software may be stored in any non-transitory computer readable medium, such as RAM, ROM, magnetic disk, or optical disk. When loaded and executed by processor(s), the processor(s) are configured to perform operations consistent with the example embodiments disclosed herein. The processor(s) may be any form of processor(s) capable of being configured to execute operations as disclosed herein. It should be understood that the terms processor and computer system or the like may be used interchangeably herein.

While this invention has been particularly shown and described with references to example embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.