Method and system for detecting targets known up to a simplex from multi-spectral and hyper-spectral imagery employing the normal compositional model

This application claims the benefit of U.S. Provisional Application No. 60/394,649, filed 9 Jul. 2002, and is related by common inventorship and subject matter to the commonly-assigned U.S. Provisional Patent Application No. 60/394,708 entitled “System and Method for Detecting Anomalies in Multispectral and Hyperspectral Imagery Employing the Normal Compositional Model” filed on 9 Jul. 2002.

Hyperspectral sensors are a new class of optical sensor that collect a spectrum from each point in a scene. They differ from multi-spectral sensors in that the number of bands is much higher (twenty or more), and the spectral bands are contiguous. For remote sensing applications, they are typically deployed on either aircraft or satellites. The data product from a hyperspectral sensor is a three-dimensional array or “cube” of data with the width and length of the array corresponding to spatial dimensions and the spectrum of each point as the third dimension. Hyperspectral sensors have a wide range of remote sensing applications including: terrain classification, environmental monitoring, agricultural monitoring, geological exploration, and surveillance. They have also been used to create spectral images of biological material for the detection of disease and other applications. Known target detection algorithms have been derived from several models of hyperspectral imagery.

The Gaussian mixture model has served as a basis for detecting known targets from hyperspectral and multispectral imagery. This approach models each datum as a realization of a random vector having one of several possible multivariate Gaussian distributions. If each observation, y∈R n , arises from one of d normal classes then the data have a normal or Gaussian mixture probability density function:

p ⁡ ( y ) = ∑ k = 1 d ⁢ ω k ⁢ N ⁡ ( μ k , Γ k ) ⁢ ( y ) , ω k ≥ 0 , ∑ k = 1 d ⁢ ω k = 1 , [Eqn.  1]

where ω k is the probability of class k and

N ⁡ ( μ k , Γ k ) ⁢ ( y ) = 1 ( 2 ⁢ ⁢ π ) n 2 ⁢  Γ k  1 2 ⁢ ⁢ exp ⁢ ⁢ ( - 1 2 ⁢ ⁢ ( y - μ k ) T ⁢ Γ k - 1 ⁡ ( y - μ k ) )

is the normal probability density function having mean μ k and covariance Γ k* . The parameters {(ω k ,μ k ,Γ k )|1≦k≦d} are typically estimated from the imagery using defined clusters, the expectation maximization algorithm or related algorithms such as the stochastic expectation maximization algorithm. Known target detection algorithms are generally implemented using a bank or a linear combination of the likelihood ratio detection statistics for each class. The covariance of the observations under the target present hypothesis is usually assumed to equal the covariance of the observations under the background only hypothesis. Thus the test for the presence of a target against background class k is often formulated as the likelihood ratio for the hypotheses:

H 0,k :y˜N (μ k ,Γ k )

H 1,k :y˜N (s,Γ k ),

where s∈R n is the spectrum of the target. In this case, the log of the likelihood ratio is equivalent to the spectral matched filter for a target against a background modeled by class k, i.e.

T MF ⁡ ( y ; k ) = ( s - μ k ) T ⁢ Γ k - 1 ⁡ ( y - μ k ) ( s - μ k ) T ⁢ Γ k - 1 ⁡ ( s - μ k ) . [Eqn.  2]

Linear and convex models have also served as the basis for formulating known target detection algorithms. In this approach the data are modeled as

H 0 :y=Wα b +η

H 1 :y=Sα t +Wα b +η,  [Eqns. 3]

where: W is an n×P matrix such that the columns of W span an interference subspace of dimension P; S is an n×Q matrix such that the columns of S span a signal subspace of dimension Q; η is additive noise such that η˜N(0,σ 2 Γ). W, S, and Γ are assumed known, and α t ∈R Q and α b ∈R P are assumed unknown. σ 2 may be known or unknown. Additionally, constraints may be placed on the coefficient vectors α t and α b , e.g

∑ i = 1 Q ⁢ a ti + ∑ i = 1 P ⁢ a bi = 1 , (c.1) a ti ≥ 0 , a bi ≥ 0. (c.2)

General procedures have not been developed for simultaneously estimating W and Γ. However, if either 1) α b is locally constant or 2) the data may be segmented into regions such that α b is essentially constant on each region, the term Wα b may be absorbed into the noise which is then modeled by η˜N(μ,Γ), where the parameters μ and Γ are estimated locally or for each segment. With W=0, Γ may be estimated from background reference data, and if Γ=I n×n , a basis for W may be estimated as the eigenvectors of a background data correlation matrix having eigenvalues greater than σ 2 , a threshold determined from the eigenspectrum of the data correlation matrix. Eqns. 3 apply a convex or linear model to the data if the constraints (c.1, c.2) are or are not imposed, respectively.

The linear models have been used by several practitioners in the art to derive likelihood ratio and generalized likelihood ratio detection statistics. See, for example, Sharf et al. [L. L. Scharf and B. Friedlander, “Matched Subspace Detectors,” IEEE Transactions on Signal Processing , Vol 42. No. 8, August 1994, pp. 2146–2157], Kraut et al. [S. Kraut, L. L. Scharf, L. T. McWhorter, “Adaptive Subspace Detectors,” IEEE Transactions on Signal Processing,” Vol. 49, No. 1, January 2001, pp. 1–16.], and Manolakis et al. [D. Manolakis, C. Siracusa, and G. Shaw, “Hyperspectral Subpixel Target Detection Using the Linear Mixing Model,” IEEE Transactions on Geoscience and Remote Sensing, Vol 39, No. 7, July 2001, pp. 1392–1409]. Likelihood ratio and generalized likelihood ratio (GLR) techniques have also been applied to the convex model. For example, Manolakis et al. showed that the GLR test when Γ=I n×n , σ 2 is unknown, and W and S are known is

T I , M ⁡ ( y ) = (  P W ⊥ ⁡ ( y )   P ( W + S ) ⊥ ⁡ ( y )  ) n , [Eqn.  4]

where P A is orthogonal projection with reference to the Euclidean inner product onto the subspace A, and A ⊥ ⊂R n is the subspace orthogonal to A.

Spectra from a class of material are often better modeled as random rather than as fixed vectors. This may be due to biochemical and biophysical variability of materials in a scene. For such data, neither the linear mixture model nor the normal mixture model is adequate, and better classification and detection results may accrue from using more accurate methods. Stocker et al. [A. D. Stocker and A. P. Schaum, “Application of stochastic mixing models to hyperspectral detection problems,” SPIE Proceedings 3071 , Algorithms for Multispectral and Hyperspectral Imagery III , S. S. Shen and A. E. Iverson eds. August 1997] propose a stochastic mixture model in which each fundamental class is identified with a normally distributed random variable, i.e.

y i = ∑ k = 1 d ⁢ a ik ⁢ ɛ k ⁢ ⁢ such ⁢ ⁢ that ⁢ ⁢ ɛ k ~ N ⁡ ( μ k , Γ k ) ; a ik ≥ 0 ; ∑ k = 1 d ⁢ a ik = 1. [Eqn.  5]

They estimate the parameters of the model by quantizing the set of allowed abundance values, and fitting a discrete normal mixture density to the data. More precisely, let Δ=1/M denote the resolution of the quantization. Then the set of allowed coefficient sequences is

A = { ( a 1 , … ⁢ , a d ) ❘ ∑ k = 1 d ⁢ a k = 1 ; a k ∈ { 0 , Δ , … ⁢ , ( M - 1 ) ⁢ Δ , 1 } }

For each {right arrow over (α)}=(α 1 , . . . ,α d )∈A, define

μ ⁡ ( a → ) = ∑ j = 1 d ⁢ a j ⁢ μ j ⁢ ⁢ and ⁢ ⁢ Γ ⁡ ( a → ) = ∑ j = 1 d ⁢ a j 2 ⁢ Γ j . [Eqn.  6]

Then the observations are fit to the mixture model

p ⁡ ( y ) = ∑ a → ∈ A ⁢ ρ a → ⁢ N ⁡ ( μ ⁡ ( a → ) , Γ ⁡ ( a → ) ) ⁢ ( y ) . [Eqn.  7]

The fitting is accomplished using a variation of the stochastic expectation maximization algorithm such that Eqn. 6 is satisfied in a least squares sense. Stocker et al. demonstrate improved classification in comparison with clustering methods using three classes, and they demonstrate detection algorithms using this model. They note, however, that the method is impractical if the data are comprised of a large number of classes or if Δ is small, as the number of elements of A, which is given by:

 A  = ( M + 1 ) ⁢ ⋯ ⁡ ( M + d - 1 ) ( d - 1 ) ! ,

becomes very large. Furthermore, quantizing the allowed abundance values leads to modeling and estimation error.

Stocker et al. used this model to develop a known target detection statistic: the finite target matched filter (FTMF). Observations of the target, t, and background, b, are represented as samples from the normal random variable t˜N(μ 1 ,Γ 1 ) and b˜N(μ 0 ,Γ 1 ), respectively. An observation that consists of a fraction (1−ƒ) of background material and ƒ of target material is then modeled as y˜N((1−ƒ)μ 0 +ƒμ 1 ,(1−ƒ) 2 Γ 0 +ƒ 2 Γ 1 )=p(y|ƒ). Stocker et al. define the FTMF as the generalized likelihood ratio test:

T FTMF ⁡ ( y ) = max f ⁢ ⁢ p ⁡ ( y ❘ f ) p ⁡ ( y ❘ f = 0 ) ; 0 ≤ f ≤ 1 , [Eqn.  8]

and a detection algorithm is achieved by applying a threshold to the values of T FTMF . A bank of FTMFs may be applied to Gaussian mixture data given by Eqns. 1 or 7.

These unresolved problems and deficiencies are clearly felt in the art and are solved by this invention in the manner described below.

A method for detecting targets comprises: a) receiving spectral data; b) using a normal compositional model for estimating background parameters from the spectral data and target components; c) estimating abundance values of classes of the normal compositional model from the background parameters and the spectral data; d) estimating target class covariance values from the spectral data, the background parameters, and the target components; e) estimating target-plus-background abundance values from the target class covariance values, the background parameters, the spectral data, and the target components; f) employing a normal compositional model for determining a likelihood ratio detection statistic from the target class covariance values, target-plus-background abundance values, spectral data, target components, background parameters, and background abundance values; and g) generating a determination output signal that represents whether an observation includes a target from the likelihood ratio detection statistic.

For a more complete understanding of this invention, reference is now made to the following detailed description of the embodiments as illustrated in the accompanying drawing, in which like reference designations represent like features throughout the several views and wherein:

FIG. 1 is a block diagram illustrating the Normal Compositional Model Known Target Detector of this invention;

FIG. 2 is a block diagram of a flowchart illustrating the background parameter estimation method of this invention; and

FIG. 3 is a block diagram illustrating the method of this invention for initializing the endmember classes.

The invention is used to detect known signatures from spectral imagery. The invention is operated as shown in FIGS. 1–3 . As shown in FIG. 1 , the major components of the invention are: estimation of the parameters of the normal compositional model (NCM) applied to the background, estimation of the covariance matrices of the target classes, estimation of the abundance values of the classes of the NCM using the background only and the background and target models, computation of the detection statistic, and application of a decision criterion to infer, based on the values of the GLR detection statistic, if there is a target of the given type at any pixel. These steps are elucidated below.

1. Normal Compositional Model (NCM)

The normal compositional model (NCM) represents each observation y i ∈R n as:

y i = c ⁢ ⁢ η + ∑ k = 1 d ⁢ a ki ⁢ ɛ k ⁢ ⁢ such ⁢ ⁢ that [Eqn.  9] 0 ≤ a ki , and c ⁢ .1 ) ∑ k = 1 d ⁢ a ki = 1 ⁢ ⁢ or c ⁢ .2 . a ) ∑ k = 1 d ⁢ a ki ≤ 1 c ⁢ .2 . b )

or other constraints, where ε k ,η∈R n are random vectors such that ε k ˜N(μ k , Γ k ), η˜N(μ 0 , Γ 0 ) and c=0,1. The number of classes used in the model of the background is d=d 0 , and the number of classes used in the background-plus-target model is d=d 1 =d 0 +d s , where d s is the number of target components. Assuming that Γ k ≠0 for all 1≦k≦d, we do not require the linear independence of the background and or target mean vectors. Constraint c.2.b may be used in place of c.2.a to account for variations in scale or as in remote sensing, scalar variations in illumination. Applied to remote sensing data, η models path radiance, additive sensor noise, and other additive terms. By choosing c=0, and constraints c.1 and c.2.a, the model reduces to the Schaum-Stocker model (Eqn. 5). Although, with this choice of parameters and constraints the present invention has advantages over the Schaum-Stocker approach because the estimation procedure does not confine the abundance values to preselected quantized values. Therefore, it is not restricted to a small number of classes and it provides more accurate estimates of class parameter and abundance values. This model reduces to the convex and linear mixing models (Eqn. 3) by choosing Γ k =0 for all 1≦k≦d and C=1, although the parameter estimation technique described below will not refine initial estimates of the μ k in this case. It does, however, provide a maximum likelihood approach to estimating the parameters of the distribution of η. Furthermore, if Γ k =0 for all 1≦k≦d and c=1, the NCM reduces to the linear or convex mixture models if the constraints c.1 and c.2.a or c.2.b are not or are imposed, respectively. Furthermore, by imposing the constraints c.2.a and α ki =0,1 for each 1≦i≦N, exactly one of α ki =1, and the model encompasses the Gaussian mixture model (Eqn. 1). Whereas specialized constraints applied to the parameters of the NCM reduce it to the older models, in general, without imposing special constraints, the NCM provides a model having higher likelihood than these alternatives.

2. Parameter Estimation

The parameter estimation module is illustrated in FIG. 2 and described below.

A. Initialization

The initialization module is depicted in FIG. 3 . The mean value of the additive term η is obtained as a virtual shade point, and the covariance of the additive term is estimated as the sample covariance of a cluster of points near η. The data are transformed using the projection operator that is zero on the subspace spanned by the set of signal spectra. Initial estimates of the background class means are obtained by applying deterministic linear unmixing techniques to the transformed data to determine a set of endmembers. Several methods of estimating endmembers are available. These estimates of the endmembers in the transformed space are pulled back to the original data space to obtain estimates of the background endmembers. These background endmembers serve as initial values of the background class mean spectra. The initial covariance of each class is obtained as the sample covariance of a cluster of points near the class mean.

B. Updating Abundance Estimates (UA)

For given parameters (μ k ,Γ k ), 1≦k≦d, and given abundances α i =(α 1i , . . . ,α di ), let

μ ⁡ ( α i ) = ∑ k = 1 d ⁢ a ki ⁢ μ k [ Eqn . ⁢ 10 ] Γ ⁡ ( α i ) = ∑ k = 1 d ⁢ ( a ki ) 2 ⁢ Γ k . [ Eqn . ⁢ 11 ]

Then, y i ˜N(μ(α i )+μ 0 ,Γ(α i )+Γ 0 ). Maximum likelihood abundance estimates are thus obtained by solving

α ^ i = ⁢ arg ( max α i ⁢ ⁢ ( 1  Γ ⁡ ( α i ) + Γ 0  0.5 ⁢ ( 2 ⁢ ⁢ π ) n 2 ⁢ ⁢ exp ( - 1 2 ⁢ ⁢ ( y i - ⁢ ( y i - μ ⁢ ⁢ ( α i ) - μ 0 ) T ⁢ ( Γ ⁡ ( α i ) + Γ 0 ) - 1 ⁢ ( y i - μ ⁡ ( α i ) - μ 0 ) ) [ Eqn . ⁢ 12 ]

subject to the constraints c.1, and c.2.a, or c.2.b, or other constraints.

C. Update Class Parameters (UP)

For given abundance estimates, the class parameters of the background,

Ω={(μ k ,Γ k )|0 ≦k≦d},

may be estimated by applying the expectation-maximization (EM) algorithm. Let

Ω r ={(μ k r ,Γ k r )|0 ≦k≦d}

denote the estimate of the parameters after the r th iteration of the EM algorithm. Given the abundance values {α ki |1≦i≦N,1≦k≦d}, define

δ i r = [ Γ r ⁡ ( α i ) + c ⁢ ⁢ Γ 0 r ] - 1 ⁢ ( y i - μ r ⁡ ( α i ) - c ⁢ ⁢ μ 0 r ) , ψ ki r = α ki ⁢ δ i r , δ _ r = 1 N ⁢ ⁢ ∑ i = 1 N ⁢ δ i r , and ⁢ ⁢ ψ _ k r = 1 N ⁢ ⁢ ∑ i = 1 N ⁢ ψ ki r .

The EM update equations are:

μ k r + 1 = μ k r + Γ k r ⁢ ψ _ k r , for ⁢ ⁢ 1 ≤ k ≤ d . [Eqn.  13a] μ 0 r + 1 = μ 0 r + Γ 0 r ⁢ δ _ r . ( c = 1 ) [Eqn.  13b] Γ k r + 1 = Γ k r - Γ k r ⁡ ( 1 N ⁢ ⁢ ∑ i = 1 N ⁢ α ki 2 ⁡ [ Γ r ⁡ ( α i ) + c ⁢ ⁢ Γ 0 r ] 1 ) ⁢ Γ k r + Γ k r ⁡ ( 1 N ⁢ ⁢ ∑ i = 1 N ⁢ ψ ki r ⁢ ψ ki r T - ψ _ ki r ⁢ ψ _ ki r T ) ⁢ Γ k r , for ⁢ ⁢ 1 ≤ k ≤ d . [Eqn.  13c] Γ 0 r + 1 = Γ 0 r - 1 N ⁢ ⁢ Γ 0 r ⁢ ⁢ ∑ i = 1 N ⁢ c 2 ⁡ [ Γ r ⁡ ( α i ) + c ⁢ ⁢ Γ 0 r ] - 1 ⁢ Γ k r + Γ 0 r ⁡ ( 1 N ⁢ ⁢ ∑ i = 1 N ⁢ δ i r ⁢ δ i r T - δ _ r ⁢ δ _ r T ) ⁢ Γ 0 r . ( c = 1 ) [Eqn.  13d]

The class parameters are updated (UP) using the expectation-maximization equations (13) and the current abundance estimates {α ki j }. Likelihood increases with each iteration of UA or UP. Thus, a sequence of parameter estimates of increasing likelihood is obtained by the application of a sequence of updates: UA,UP,UA,UP, . . . . The iteration is halted when a convergence criterion is satisfied.

3. Detection Algorithms

Given parameters of the background and the target classes, the generalized likelihood ratio test may be computed. Let α i0 and α i1 denote the abundance estimates obtained by solving (Eqn. 12) using only the background classes and the combination of background and target classes, respectively. The log of the likelihood function given the abundance and parameter values is

L ⁡ ( y i ⁢  α ij ) = - 1 2 ⁢ ⁢ log ⁡ (  Γ ⁡ ( α ij ) + Γ 0  ) - n 2 ⁢ ⁢ log ⁢ ⁢ ( 2 ⁢ ⁢ π ) - 1 2 ⁢ ( y ij - μ ⁡ ( α ij ) - μ 0 ) t ⁢ ( Γ ⁢ ( α ij ) + Γ 0 ) - 1 ⁢ ( y ′ - μ ⁡ ( α ij ) - μ 0 ) , j = 0 , 1 , [Eqn.  14]

and the generalized likelihood ratio is

T K ⁡ ( y i ) = L ⁡ ( y i ⁢  α i1 ) L ⁡ ( y i ⁢  α i0 ) . [ Eqn . ⁢ 15 ]

An inference concerning the presence of a target in pixel i is made based on the values of T K (y i ). A threshold τ is determined that corresponds to an attribute of the test, e.g., probability of false alarm, by analyzing the probability distribution of T K (y i ) applied to background data. The decision criterion is then

T K ⁡ ( y i ) ≤ > ⁢ τ H 0 H 1 . [ Eqn . ⁢ 16 ]

The class parameters may be updated using a segmented expectation maximization algorithm in place of the expectation-maximization algorithm. In this approach a lower threshold, possibly zero, is placed on the abundance of a class, and only those pixels for which the abundance exceeds the threshold are utilized in the update of the associated class parameters. This approach saves computations and improves the speed of convergence of the parameter estimates.

Rather than solving for the maximum likelihood value of the abundance estimates in the parameter estimation phase of the operation, random samples of the abundance estimates may be generated and these may be used in place of the maximum likelihood estimates in the updating of the class parameters.

For parameter estimation, the image may be culled of spectra that are sufficiently close to the target spectra in order to further reduce the bias in the estimate of background parameters from data that may also contain targets.

The processing may be conceived of as applying two models to the data: 1) the background only model, and 2) the background and target model. In the description above, the background parameters were estimated only once. Alternatively, one could estimate parameters of the background only model and separately estimate parameters of the background-plus-target model.

Referring to FIG. 1 , there is shown a block diagram of an embodiment of a process 100 for detecting targets. Spectral data 112 that may be generated at step 110 is received at step 120 which uses a normal compositional model, as defined above, for estimating background parameters 122 from the spectral data 112 and target components 116 . Next, step 130 estimates abundance values 132 of classes of the normal compositional model from the background parameters 122 and spectral data 112 . Target class covariance values 152 are estimated at step 150 from the spectral data 112 , background parameters 122 , and target components 116 . Then, step 160 estimates target-plus-background abundance values 162 from the target class covariance values 152 , background parameters 122 , spectral data 112 , and target components 116 . A normal compositional model is employed at step 140 for determining a likelihood ratio detection statistic 142 from the target class covariance values 152 , target-plus-background abundance values 162 , spectral data 112 , target components 116 , background parameters 122 , and background abundance values 132 . Finally, at step 170 , a determination output signal 172 is generated from the likelihood ratio detection statistic 142 that represents whether an observation includes a target.

Referring to FIG. 2 , step 120 for estimating background parameters 122 is further shown to include step 220 for initializing current class parameters 222 and current abundance estimates 224 . Then, step 230 defines updated abundance estimates 232 from the current class parameters 222 and spectral data 112 . Step 250 determines converged class parameter candidates 252 from the updated abundance estimates 232 , current class parameters 222 , and spectral data 112 . At step 270 , the background parameters 122 are generated if the converged class parameter candidates 252 satisfy first convergence criteria. If the first converged class parameters 252 do not satisfy the first convergence criteria, then step 270 generates a non-convergence signal 274 that is provided to step 230 , to which step 120 returns.

Step 250 for determining converged class parameter candidates 252 further includes step 240 for creating updated background class parameters 242 from the current class parameters 222 , updated abundance estimates 232 , and spectral data 112 , and step 260 for generating converged class parameter candidates 252 if the background class parameters 242 satisfy second convergence criteria. However, if the background class parameters 242 do not satisfy second convergence criteria, then step 260 generates a non-convergence signal that is provided to step 240 , to which step 250 returns.

FIG. 3 further illustrates step 220 for initializing the current class parameters 222 which is shown to include: a) step 310 for defining a shade point offset value 312 ; b) step 320 for defining a shade point covariance value 322 from the shade point offset value 312 and spectral data 112 ; c) step 330 for generating reduced spectral data 332 from the spectral data 112 , target components 116 , and shade point offset value 312 ; d) step 340 for defining end members 342 from the reduced spectral data 332 ; and e) step 350 for generating the current class parameters 222 from the end members 342 and spectral data 112 .

Clearly, other embodiments and modifications of this invention may occur readily to those of ordinary skill in the art in view of these teachings. Therefore, this invention is to be limited only by the following claims, which include all such embodiments and modifications when viewed in conjunction with the above specification and accompanying drawing.