Optimized stochastic resonance method for signal detection and image processing

This application claims priority to U.S. Provisional Patent Application No. 60/728,504, filed Oct. 20, 2006.

The U.S. Government has a paid-up license in this invention and the right in limited circumstances to require the patent owner to license others on reasonable terms as provided for by the terms of Contract No. FA9550-05-C-0139 awarded by the Air Force Office of Scientific Research (AFOSR).

1. Field of the Invention

The present invention relates to signal detection and, more particularly, to a method of detecting signals and processing images using stochastic resonance.

2. Description of the Related Art

Stochastic resonance (SR) is a nonlinear physical phenomenon in which the output signals of some nonlinear systems can be enhanced by adding suitable noise under certain conditions. The classic SR signature is the signal-to-noise ratio (SNR) gain of certain nonlinear systems, i.e., the output SNR is higher than the input SNR when an appropriate amount of noise is added.

Although SNR is a very important measure of system performance, SNR gain-based SR approaches have several limitations. First, the definition of SNR is not uniform and it varies from one application to another. Second, to optimize the performance, the complete a priori knowledge of the signal is required. Finally, for detection problems where the noise is non-Gaussian, SNR is not always directly related to detection performance; i.e., optimizing output SNR does not guarantee optimizing probability of detection.

In signal detection theory, SR also plays a very important role in improving the signal detectability. For example, improvement of detection performance of a weak sinusoid signal has been reported. To detect a DC signal in a Gaussian mixture noise background, performance of the sign detector can be enhanced by adding some white Gaussian noise under certain circumstances. For the suboptimal detector known as the locally optimal detector (LOD), detection performance is optimum when the noise parameters and detector parameters are matched. The stochastic resonance phenomenon in quantizers results in a better detection performance can be achieved by a proper choice of the quantizer thresholds. Detection performance can be further improved by using an optimal detector on the output signal. Despite the progress achieved by the above approaches, the use the SR effect in signal detection systems is rather limited and does not fully consider the underlying theory of SR.

Simple and robust suboptimal detectors are used in numerous applications. To improve a suboptimal detector detection performance, two approaches are widely used. In the first approach, the detector parameters are varied. Alternatively, when the detector itself cannot be altered or the optimum parameter values are difficult to obtain, adjusting the observed data becomes a viable approach. Adding a dependent noise is not always possible because pertinent prior information is usually not available.

For some suboptimal detectors, detection performance can be improved by adding an independent noise to the data under certain conditions. For a given type of SR noise, the optimal amount of noise can be determined that maximizes the detection performance for a given suboptimal detector. However, despite the progress made, the underlying mechanism of the SR phenomenon as it relates to detection problems has not fully been explored. For example, until now the “best” noise to be added in order to achieve the best achievable detection performance for the suboptimal detector was not known. Additionally, the optimal level of noise that should be used for enhanced performance was also unknown.

It is therefore a principal object and advantage of the present invention to provide a method for determining the best noise to add to improve detection of a suboptimal, non-linear detector.

It is an additional object and advantage of the present invention to provide a method for determining the optimal level of noise for improved detection.

In accordance with the foregoing objects and advantages, the present invention provides a method for signal detection in observed sensor data for a broad range of electromagnetic or acoustic applications such as radar, sonar, as well as imagery such as visual, hyperspectral, and multi-spectral. The method of the present invention is applicable in applications involving non-linear processing of the data. Specifically, the method of the present invention determines the stochastic resonance noise probability density function to be added to either the observed data process to optimize detection with no increase in the false alarm rate, or to an image to optimize the detection of signal objects from the background. In addition, the method of the present invention determines the conditions required for performance improvement using additive stochastic resonance noise. The method of the present invention also yields a constant false alarm rate (CFAR) receiver implementation, which is essential in operational conditions in which it is imperative to maintain false alarm rates without adjusting the detector threshold level.

The present invention will be more fully understood and appreciated by reading the following Detailed Description in conjunction with the accompanying drawings, in which:

FIG. 1 is a graph of the effect of additive noise according to the present invention.

FIG. 2 is a graph of the values of F 1 and F 0 as a function of x according to the present invention.

FIG. 3 is a graph of the relationship between f 1 and f 0 according to the present invention.

FIG. 4 is a graph of the relationship between G(f 0 ; k), f 0 , f 0i (k), v i (k) with i=1, 2 and different k value 0, 1 and 2 according to the present invention.

FIG. 5 is a graph illustrating different H(x) curves where μ=3, A=1 according to the present invention.

FIG. 6 is a graph of P D y as a function of signal level A in Gaussian mixture noise when μ=3 and σ 0 =1 according to the present invention.

FIG. 7 is a graph of P D y as a function of σ 0 for different types of noise enhanced detectors when μ=3 and A=1 according to the present invention.

FIG. 8 is a graph of P D y as a function of μ for different types of noise enhanced detectors when σ 0 =1 and A=1 according to the present invention.

FIG. 9 is a graph of the ROC curves for different SR noise enhanced sign detectors when N=30 according to the present invention.

FIG. 10 is a schematic of an SR detection system according to the present invention.

FIG. 11 is a schematic of an SR detection system according to the present invention

Referring now to the drawings, wherein like reference numerals refer to like parts throughout, there is seen in FIG. 1 a chart illustrating the effective of additive noise on a given signal.

The following definitions serve to clarify the present invention:

The term “constant false alarm rate” (CFAR) refers to the attribute of a receiver that maintains the false alarm rate fixed in the presence of changing interference levels.

The term “false alarm” refers to the decision that a signal is present when in fact it is not.

The term “false alarm rate” refers to the rate at which a false alarm occurs.

The term “fixed detector” refers to a detector comprised of a fixed test statistic and a fixed threshold.

The term “receiver operating characteristic” (ROC) refers to a plot of the probability of detection as a function of the probability of false alarm for a given detector.

To enhance the detection performance, noise is added to an original data process x to obtain a new data process y given by y=x+n, where n is either an independent random process with pdf p n (•) or a nonrandom signal. There is no constraint on n. For example, n can be white noise, colored noise, or even be a deterministic signal A, corresponding to p n (n)=δ(n−A). As described herein, depending on the detection problem, an improvement of detection performance may not always be possible. In that case, the optimal noise is equal to zero. The pdf of y is expressed by the convolutions of the pdfs such that

p y ( y )= p x ( x )* p n ( x )=∫ R N p x ( x ) p n ( y−x ) dx )  (7)

The binary hypotheses testing problem for this new observed data y can be expressed as:

Since the detector is fixed, i.e., the critical function φ of y is the same as that for x, the probability of detection based on data y is given by,

Alternatively,

Similarly,

P FA y = ∫ R N ⁢ p 0 ⁡ ( x ) ⁢ C n , ϕ ⁡ ( x ) ⁢ ⅆ x = E 0 ⁡ [ C n , ϕ ⁡ ( x ) ] ( 12 ) ⁢ = ∫ F 0 , ϕ ⁡ ( x ) ⁢ p n ⁡ ( x ) ⁢ ⅆ x = E n ⁡ ( F 0 , ϕ ⁡ ( x ) ) ⁢

⁢ where ( 13 ) F i , ϕ ⁡ ( x ) ≡ ∫ R N ⁢ ϕ ⁡ ( y ) ⁢ p i ⁡ ( y - x ) ⁢ ⅆ y ⁢ ⁢ i = 0 , 1 , ( 14 )

corresponding to hypothesis H i . E i (•), E n (•) are the expected values based on distributions p i and p n , respectively, and P x FA =F 0,φ (0), P x D =F 1,φ (0). To simplify notation, subscript φ of F and C may be omitted and denotes as F 1 , F 0 , and C n , respectively. Further, F 1 (x 0 ) and F 0 (x 0 ) are actually the probability of detection and probability of false alarm, respectively, for this detection scheme with input y=x+x 0 . For example, F 1 (−2) is the P D of this detection scheme with input x−2. Therefore, it is very convenient to obtain the F 1 and F 0 values by analytical computation if p 0 , p 1 and φ are known. When they are not available, F 1 and F 0 can be obtained from the data itself by processing it through the detector and recording the detection performance. The optimal SR noise definition may be formalized as follows.

Consider the two hypotheses detection problem. The pdf of optimum SR noise is given by

Conditions 1) and 2) are fundamental properties of a pdf function. Condition 3) ensures that P y FA ≦P x FA , i.e., the P FA constraint specified under the Neyman-Pearson Criterion is satisfied. Further, if the inequality of condition 3) becomes equality, the Constant False Alarm Rate (CFAR) property of the original detector is maintained. A simple illustration of the effect of additive noise is shown in FIG. 1 . In the example,

F 1 ⁡ ( - A ) = max x ⁢ F 1 ⁡ ( x ) ⁢ ⁢ and ⁢ ⁢ F 0 ⁡ ( - A ) < F 0 ⁡ ( 0 ) ,

hence

p n opt =δ( x+A )

which means the optimal SR noise n=−A is a dc signal with value −A. In practical applications, some additional restrictions on the noise may also be applied. For example, the type of noise may be restricted, (e.g., n may be specified as Gaussian noise), or we may require a noise with even symmetric pdf p n (x)=p n (−x) to ensure that the mean value of y is equal to the mean value of x. However, regardless of the additional restrictions, the conditions 1), 2), and 3) are always valid and the optimum noise pdf can be determined for these conditions.

In general, for optimum SR noise detection in Neyman-Peason detection, it is difficult to find the exact form of p n (•) directly because of condition 3). However, an alternative approach considers the relationship between p n (x) and F i (x). From equation (14), for a given value f 0 of F 0 , we have x=F 0 −1 (f 0 ), where F 0 −1 is the inverse function of F 0 . When F 0 is a one-to-one mapping function, x is a unique vector. Otherwise, F 0 −1 (f 0 ) is a set of x for which F 0 (x)=f 0 . Therefore, we can express a value or a set of values f 1 of F 1 as

ƒ 1 =F 1 ( x )= F 1 ( F 0 −1 (ƒ 0 ))  (16)

Given the noise distribution of p n (•) in the original RN domain, p n,ƒ 0 (•), the noise distribution in the f 0 domain can also be uniquely determined. Further, the conditions on the optimum noise can be rewritten in terms of f 0 equivalently as

4 ) p n , f 0 ⁡ ( f 0 ) ≥ 0 5 ) ∫ p n , f 0 ⁡ ( f 0 ) ⁢ ⅆ f 0 = 1 6 ) ∫ f 0 ⁢ p n , f 0 ⁡ ( f 0 ) ⁢ ⅆ f 0 ≤ P FA x and P D y = ∫ 0 1 ⁢ f 1 ⁢ p n , f 0 ⁡ ( f 0 ) ⁢ ⅆ f 0 , ( 17 )

where p n,ƒ 0 (•) is the SR noise pdf in the f 0 domain.

Compared to the original conditions 1), 2) and 3), this equivalent form has some advantages. First, the problem complexity is dramatically reduced. Instead of searching for an optimal solution in R N , the present invention seeks an optimal solution in a single dimensional space. Second, by applying these new conditions, the present invention avoids the direct use of the underlying pdfs p 1 (•) and p 0 (•) and replace them with f 1 and f 0 . Note that, in some cases, it is not very easy to find the exact form of f 0 and f 1 . However, recall that F 1 (x 0 ) and F 0 (x 0 ) are the Probability of Detection and Probability of False Alarm, respectively, of the original system with input x+x 0 . In practical applications, the relationship may be determined by Monte Carlo simulation using importance sampling. In general, compared to p 1 and p 0 , f 1 and f 0 are much easier to estimate and once the optimum p n,ƒ 0 is found, the optimum p n (x) is determined as well by the inverse of the functions F 0 and F 1 .

Consider the function J(t), such that J(t)=sup(f 1 :f 0 =t) is the maximum value of f 1 given f 0 . Clearly, J(P x FA )≧F 1 (0)=P x D . It follows that for any noise p n ,

P D y ( p n )=∫ 0 1 J ( f 0 ) p n,ƒ 0 (ƒ 0 ) dƒ 0   (18)

Therefore, the optimum P D y is attained when f 1 (f 0 )=J(f 0 ) and P D,opt y =E n (J).

Improvability of a given detector when SR noise is added can be determined by computing and comparing P D,opt y and P D x . When P D,opt y >P D x , the given detector is improvable by adding SR noise. However, it requires the complete knowledge of F 1 (•) and F 0 (•) and significant computation. For a large class of detectors, however, depending on the specific properties of J, it is possible to determine the sufficient conditions for improvability and non-improvability more easily. The conditions are determined using the following theorems.

Theorem 1 (Improvability of Detection via SR): If J(P FA x )>P D x or J″(P FA x )>0 when J(t) is second order continuously differentiable around P FA x , then there exists at least one noise process n with pdf p n (•) that can improve the detection performance.

Proof: First, when J(P FA x )>P D x , from the definition of J function, we know that there exists at one least one n 0 such that F 0 (n 0 )=P FA x and F 1 (n 0 )=J(P FA x )>P D x . Therefore, the detection performance can be improved by choosing a SR noise pdf P n (n)=δ(n−n 0 ). When J″(P FA x )>0 and is continuous around P FA x , there exists an ε>0 such that J″(•)>0 on I=(P FA x −ε, P FA x +ε). Therefore, from Theorem A-1, J is convex on I. Next, add noise n with pdf p n (x)=½δ(x−x 0 )+½δ(x+x 0 ), where F 0 (x 0 )=P FA x +ε/2 and F 0 (x 1 )=P FA x −ε/2. Due to the convexity of J,

P D y = J ⁡ ( P FA x - ɛ 2 ) + J ⁡ ( P FA x - ɛ 2 ) 2 > J ⁡ ( P FA x ) ≥ P D x

Thus, detection performance can be improved via the addition of SR noise.

Theorem 2 (Non-improvability of Detection via SR): If there exists a non-decreasing concave function Ψ(f 0 ) where Ψ(P x FA )=J(P x FA )=F 1 (0) and Ψ(f 0 )≧J(f 0 ) for every f 0 , then P D y ≦P D x for any independent noise, i.e., the detection performance cannot be improved by adding noise.

Proof: For any noise n and corresponding y, we have

P D y ⁡ ( p n ) = ⁢ ∫ 0 1 ⁢ J ⁡ ( f 0 ) ⁢ p n , f 0 ⁡ ( f 0 ) ⁢ ⅆ f 0 ≤ ⁢ ∫ 0 1 ⁢ Ψ ⁡ ( f 0 ) ⁢ p n , f 0 ⁡ ( f 0 ) ⁢ ⅆ f 0 ≤ ⁢ Ψ ⁡ ( ∫ 0 1 ⁢ f 0 ⁢ p n , f 0 ⁡ ( f 0 ) ⁢ ⅆ f 0 ) ≤ ⁢ Ψ ⁡ ( P FA x ) = P D x ( 19 )

The third inequality of the Right Hand Side (RHS) of (19) is obtained using the concavity of the Ψ function. The detection performance cannot be improved via the addition of SR noise.

Before determining the form of the optimum SR noise PDF, i.e., the exact pdf of p n opt , the following result for the form of optimum SR noise must be determined.

Theorem 3 (Form of Optimum SR Noise): To maximize P D y , under the constraint that P FA y ≦P FA x , the optimum noise can be expressed as:

p n opt ( n )=λδ( n−n 1 )+(1−λ)δ( n−n 2 )  (20)

where 0≦λ≦1. In other words, to obtain the maximum achievable detection performance given the false alarm constraints, the optimum noise is a randomization of two discrete vectors added with the probability λ and 1−λ, respectively.

Proof: Let U={(f 1 , f 0 )|f 1 =F 1 (x), f 0 =F 0 (x), x R N ) be the set of all pairs of (f 1 ; f 0 ). Since 0≦f 1 ; f 0 ≦1, U is a subset of the linear space R 2 . Furthermore, let V be the convex hull of U. Since V⊂R 2 , its dimension Dim(V)≦2. Similarly, let the set of all possible (P D y ; P FA y ) be W. Since any convex combination of the elements of U, say

⁢ ( χ , ϕ ) = ∑ i = 1 M ⁢ α i ⁡ ( f 1 , i , f 0 , i )

can be obtained by setting the SR noise pdf such that

p n , f 0 ⁡ ( f 0 ) = ∑ i = 1 M ⁢ α i ⁢ δ ⁡ ( f 0 - f 0 , i )

V ⊂ W. It can also be shown that W ⊂ V. Otherwise, there would exist at least one element z such that z W, but z∉V. In this case, there exists a small set S and a positive number τ such that

S ={( x,y )|∥( x,y )− z ∥ 2 2 <τ} and S∩V=‘{ }’

where ‘{ }’ denotes an empty set. However, since 0≦f 1 ; f 0 ≦1, by the well known property of integration, there always exists a finite set E with finite elements such that E ⊂ U and (x 1 ; y 1 ), a convex combination of the elements of E, such that

∥( x 1 ,y 1 )− z∥ 2 2 <τ

Since (x 1 ; y 1 ) V, then (x 1 ; y 1 ) (V∩S) which contradicts the definition of S. Therefore, W ⊂ V. Hence, W=V. From Theorem A-4, (P y D ; P y FA ) can be expressed as a convex combination of three elements. Also, since we are only interested in maximizing P D under the constraint that P FA y ≦P FA x , the optimum pair can only belong to B, the set of the boundary elements of V. To show this, let (f 1 *; f 0 *) be an arbitrary non-boundary point inside V. Since there exists a τ>0 such that (f 1 *+τ, f 0 *) V, then (f 1 *; f 0 *) is inadmissible as an optimum pair. Thus, the optimum pair can only exist on the boundary and each z on the boundary of V can be expressed as the convex combination of only two elements in U. Hence,

( P D,opt y ,P FA,opt y )=λ(ƒ 11 ,ƒ 01 )+(1−λ)(ƒ 12 ,ƒ 02 )  (21)

where (f 11 ; f 01 ); (f 12 ; f 02 ) U, 0≦λ≦1. Therefore, we have

p n,ƒ 0 opt =λδ(ƒ 0 −ƒ 01 )+(1−λ)δ(ƒ 0 −ƒ 02 )  (22)

Equivalently, p n opt (n)=λδ(n−n 1 )+(1−λ)δ(n−n 2 ), where n 1 and n 2 are determined by the equations

Alternatively, the optimum SR noise can also be expressed in terms of C n , such that

C n opt ( x )=λφ( x+n 1 )+(1−λ)φ( x+n 2 )  (24)

From equation (22), we have

P D,opt y =λJ (ƒ 01 )+(1−λ) J (ƒ 02 )  (25)

and

P FA,opt y =λƒ 01 +(1−λ)ƒ 02 ≦P FA x   (26)

Depending on the location of the maxima of J(•), determination of the pdf of optimum SR noise may be accomplished according to the following theorem.

Theorem ⁢ ⁢ 4 ⁢ : Let ⁢ ⁢ F 1 ⁢ M = max ⁡ ( J ⁡ ( t ) ) ⁢ ⁢ and ⁢ ⁢ t 0 = arg ⁢ ⁢ min t ⁢ ( J ⁡ ( t ) = F 1 ⁢ M ) .

It follows that

Case 1: If t o ≦P FA x , then P FA, opt x =t o and P D, opt y =F 1M , i.e., the maximum achievable detection performance is obtained when the optimum noise is a DC signal with value no, i.e.,

p n opt ( n )=δ( n−n 0 )  (27)

where F 0 (n o )=t o and F 1 (n o )=F 1M .

Case 2: If t o >P FA x , then P FA,opt x =F 0 (0)=P FA x , i.e., the inequality of (26) becomes equality. Furthermore,

P FA,opt y =λƒ 01 +(1−λ)ƒ 02 =P FA x   (28)

Proof: For Case 1, notice that

P D y = ∫ 0 1 ⁢ J ⁡ ( f 0 ) ⁢ p n , f 0 ⁡ ( f 0 ) ⁢ ⅆ f 0 ≤ ∫ 0 1 ⁢ F 1 ⁢ M ⁢ p n , f 0 ⁡ ( f 0 ) ⁢ ⅆ f 0 = F 1 ⁢ M

and F 1 (n 0 )=F 1M . Therefore the optimum detection performance is obtained when the noise is a DC signal with value n 0 with P FA y =t 0 .

The contradiction method is used to prove Case 2. First, supposing that the optimum detection performance is obtained when P FA,opt y =κ<P FA x with noise pdf p n,ƒ 0 opt (ƒ 0 ).

Let ⁢ ⁢ p n 1 , f 0 ⁡ ( f 0 ) = P FA x - κ t 0 - κ ⁢ δ ⁡ ( f 0 - t 0 ) + t 0 - P FA x t 0 - κ ⁢ p n , f 0 opt ⁡ ( f 0 ) .

It is easy to verify that p n,f 0 (f 0 ) is a valid pdf. Let y 1 =x+n1. We now have

,

⁢

and

=

But this contradicts (15), the definition of p n opt . Therefore, P FA,opt y =P FA x , i.e., the maximum achievable detection performance is obtained when the probability of false alarm remains the same for the SR noise modified observation y.

For Case 2 of Theorem 4, i.e., when t 0 >P FA x , let us consider the following construction to derive the form of the optimum noise pdf. From Theorem 4, we have the condition that P FA,opt y =F 0 (0)=P FA x is a constant. Define an auxiliary function G such that

G (ƒ 0 ,k )= J (ƒ 0 )− kƒ 0 ,  (29)

where k R. We have P D y =E n (J)=E n (G(f 0 , k))+kE n (f 0 )=E n (G(f 0 ; k))+k P FA x . Hence, p n,ƒ 0 opt also maximizes E n (G(f 0 ; k)) and vice versa. Therefore, under the condition that P FA y =P FA x , maximization of P D y is equivalent to maximization of E n (G(f 0 ; k)). Divide the domain of f 0 into two intervals I 1 =[0, P FA x ] and I 2 =[P FA x , 1]. Let f 01 (k) be the minimum value that maximizes G(f 0 ; k) in I 1 and let f 02 (k) be the minimum value that maximizes G(f 0 ; k) in I 2 . Also, let v 1 (k)=G(f 01 ; k) and v 2 (k)=G(f 02 ; k) be the corresponding maximum values. Since for any f 0 , G(f 0 ; k) is monotonically decreasing when k is increasing, v 1 (k) and v 2 (k) are monotonically decreasing while f 01 (k) and f 02 (k) are monotonically non-increasing when k is increasing. Since G(f 0 ; 0)=J, therefore v 2 (0)=F 1M >v 1 (0), furthermore, when k is very large, we have v 1 (k)=J(0)>v 2 (k) =J(P FA x )−kP FA x . Hence, there exists at least one k 0 >0 such that v 1 (k 0 )=v 2 (k 0 )≡v. For illustration purposes, the plots of G(f 0 ; k) for the detection problem discussed below are shown in FIG. 4 . Divide the [0,1] interval into two non-overlapping parts A, {f 01 (k 0 ), f 02 (k 0 )}, such that {f 01 (k 0 ); f 02 (k 0 )}∪A=[0,1] and {f 01 (k 0 ); f 02 (k 0 )}∩A={ }. Next, represent p n, f 0 (f 0 ) as

p n,f 0 (ƒ 0 )=α 1 δ(ƒ 0 −ƒ 01 ( k 0 ))+α 2 δ(ƒ 0 −ƒ 02 ( k 0 ))+ I A (ƒ 0 ) p n,ƒ 0 (ƒ 0 )  (30)

where I A (f 0 )=1 for f 0 A and is zero otherwise (an indicator function). From equation (5), we must have

Note that J(f 0 )≦v for all f 0 A. Clearly, the upper bound can be attained when p n,ƒ 0 =0 for all f 0 A, i.e., α 1 +α 2 =1. Therefore, P D,opt y P D,opt y =E n (G)+k 0 P FA x =v+k 0 P FA x . From equation (28), we have

Notice that by letting

λ = f 02 ⁡ ( k 0 ) - P FA x f 02 ⁡ ( k 0 ) - f 01 ⁡ ( k 0 ) ,

(33) is equivalent to (22).

Equivalently, we have the expression of p n opt (n) as

Further, in the special case where f 1 is continuously differentiable, G is also continuously differentiable. Since f 01 and f 02 are at least local maxima, we have

=

Therefore, from the derivative of (29), we have

In other words, the line connecting (J(f 01 (k 0 )),f 01 (k 0 )) and J(f 02 (k 0 )),f 02 (k 0 )) is the bi-tangent line of J(•) and k 0 is its slope. Also,

P D,opt y =v+k 0 P FA x   (37)

Thus, the condition under which SR noise can improve detection performance has been derived, and the specific form of the optimum SR noise has been obtained.

Detection Example

In a detection problem where two hypotheses H0 and H1 are given as

{ H 0 ⁢ x ⁡ [ i ] = ω ⁡ [ i ] H 1 ⁢ x ⁡ [ i ] = A + ω ⁡ [ i ] , ( 38 )

for i=0, 1, . . . , N−1, A>0 is a known dc signal, and w[i] are i.i.d noise samples with a symmetric Gaussian mixture noise pdf as follows

p w ⁡ ( w ) = 1 2 ⁢ γ ⁡ ( w ; - μ , σ 0 2 ) + 1 2 ⁢ γ ⁡ ( w ; μ , σ 0 2 ) ⁢

⁢ and ⁢ ⁢ where ⁢

⁢ γ ⁡ ( w ; μ , σ 0 2 ) = 1 2 ⁢ πσ 2 ⁢ exp ⁡ [ - ( w - μ ) 2 2 ⁢ σ 2 ] ( 39 )

setting μ=3, A=1 and σ 0 =1. A suboptimal detector is considered with test statistic

From equation (40), this detector is essentially a fusion of the decision results of N i.i.d. sign detectors.

When N=1, the detection problem reduces to a problem with the test statistic T 1 (x)=x, thresholds η=0 (sign detector) and the probability of false alarm P FA x =0.5. The distribution of x under the H 0 and H 1 hypotheses can be expressed as

p 0 ⁡ ( x ) = 1 2 ⁢ γ ⁡ ( x ; - μ , σ 0 2 ) + 1 2 ⁢ γ ⁡ ( x ; μ , σ 0 2 ) ⁢

⁢ and ( 41 ) p 1 ⁡ ( x ) = 1 2 ⁢ γ ⁡ ( x ; - μ + A , σ 0 2 ) + 1 2 ⁢ γ ⁡ ( x ; μ + A , σ 0 2 ) ( 42 )

respectively. The critical function is given by

The problem of determining the optimal SR noise is to find the optimal p(n) where for the new observation y=x+n, the probability of detection P D y =p(y>0;H 1 ) is maximum while the probability of false alarm P FA y =p(y>0;H 0 )≦P FA x =½.

When N>1, the detector is equivalent to a fusion of N individual detectors and the detection performance monotonically increases with N. Like the N=1 case, when the decision function is fixed, the optimum SR noise can be obtained by a similar procedure. Due to space limitations, only the suboptimal case where the additive noise n is assumed to be an i.i.d noise is considered here. Under this constraint, since the P D S and P FA S of each detector are the same, it can be shown that the optimal noise for the case N>1 is the same as N=1 because P FA ≦0.5 is fixed for each individual detector while increasing its P D . Hence, only the one sample case (N=1) is considered below. However, the performance of the N>1 case can be derived similarly.

The determination of the optimal SR noise pdf follows from equations (11) and (13), where it can be shown that in this case,

It is also easy to show that, in this case, F 1 (x)>F 0 (x) and both are monotonically increasing with x. Therefore, J(f 0 )=f 1 (f 0 )=f 1 , and U=(f 1 ; f0) is a single curve. FIG. 2 shows the values of f 1 and f 0 as a function of x, while the relationship between F 1 and F 0 is shown in FIG. 3 . V, the convex hull of all possible P D and P FA after n is added and shown as the light and dark shadowed regions in FIG. 3 . Note that a similar non-concave ROC occurs in distributed detection systems and dependent randomization is employed to improve system performance. Taking the derivative of f 1 w.r.t. f 0 , we have

ⅆ ( f 1 ) ⅆ ( f 0 ) = ⅆ ( f 1 ) ⅆ ( x ) ⅆ ( f 0 ) ⅆ ( x ) = p 1 ⁡ ( - x ) p 0 ⁡ ( - x ) ,

⁢ and ( 46 ) ⅆ 2 ⁢ ( f 1 ) ⅆ ( f 0 2 ) = ⁢ 1 p 0 ⁡ ( - x ) ⁢ ⅆ ( p 1 ⁡ ( - x ) p 0 ⁡ ( - x ) ) ⅆ x = ⁢ - p 1 ′ ⁡ ( - x ) ⁢ p 0 ⁡ ( - x ) + p 0 ′ ⁡ ( - x ) ⁢ p 1 ⁡ ( - x ) p 0 3 ⁡ ( - x ) ( 47 )

where x=F 0 −1 (f 0 ). Since

ⅆ γ ⁡ ( y - x ; μ , σ 2 ) ⅆ x = μ - x σ 2 ⁢ γ ⁡ ( y - x ; μ , σ 2 )

we have p′ 0 (−x)| x=0 and

With respect to the improvability of this detector, when A<μ, setting (48) equal to zero and solving the equation for σ 0 , we have σ 1 , the zero pole of (48)

σ

1

=

When σ 0 <σ 1 , then

ⅆ 2 ⁢ ( f 1 ) ⅆ ( f 0 2 ) ⁢ | f 0 = F 0 ⁡ ( 0 ) ⁢ > 0

and, in this example, σ 1 2 =8.6562>σ 0 2 =1. From Theorem 1, this detector is improvable by adding independent SR noise. When

A > μ , ⅆ 2 ⁢ ( f 1 ) ⅆ ( f 0 2 ) ⁢ | f 0 = f 0 ⁡ ( 0 ) ⁢ < 0

the improvability cannot be determined by Theorem 1. However, for this particular detector, as discussed below, the detection performance can still be improved.

The two discrete values as well as the probability of their occurrence may be determined by solving equations (35) and (36). From equations (44) and (45), the relationship between f 1 , f 0 and x, and equation (46), we have (49)

Although it is generally very difficult to solve the above equation analytically, in this particular detection problem,

p 1 ⁡ ( - ( μ - A 2 ) ) = 0.5 ⁢ γ ⁡ ( - A 2 ; 0 , σ 0 2 ) + 0.5 ⁢ γ ⁡ ( 2 ⁢ μ + A 2 ; 0 , σ 0 2 ) ,

⁢ p 0 ⁡ ( - ( μ - A 2 ) ) = 0.5 ⁢ γ ⁡ ( - A 2 ; 0 , σ 0 2 ) + 0.5 ⁢ γ ⁡ ( 2 ⁢ μ - A 2 ; 0 , σ 0 2 ) ,

⁢ p 1 ⁡ ( - ( μ - A 2 ) ) = 0.5 ⁢ γ ⁡ ( - A 2 ; 0 , σ 0 2 ) + 0.5 ⁢ γ ⁡ ( 2 ⁢ μ - A 2 ; 0 , σ 0 2 ) ,

⁢ p 0 ⁡ ( - ( μ - A 2 ) ) = 0.5 ⁢ γ ⁡ ( - A 2 ; 0 , σ 0 2 ) + 0.5 ⁢ γ ⁡ ( 2 ⁢ μ + A 2 ; 0 , σ 0 2 ) ,

⁢ so ⁢ ⁢ that p 1 ⁡ ( - ( μ - A 2 ) ) p 0 ⁡ ( - ( μ - A 2 ) ) ≅ 1 , p 1 ⁡ ( - ( - μ - A 2 ) ) p 0 ⁡ ( - ( - μ - A 2 ) ) ≅ 1 and F 1 ⁡ ( ( μ - A 2 ) ) - F 1 ⁡ ( ( - μ - A 2 ) ) = F 0 ⁡ ( ( μ - A 2 ) ) - F 0 ⁡ ( ( - μ - A 2 ) )

given 2μ−A/2>3σ 0 . Thus, the roots n 1 ; n 2 of equation (49) can be approximately expressed as n 1 =−μ−A/2 and n 2 =μ−A/2.

Correspondingly,

The present invention also encompasses special cases where the SR noise is constrained to be symmetric. These include symmetric noise with arbitrary pdf p s (x), white Gaussian noise p g (x)=γ(x; 0, σ 2 ) and white uniform noise p u (x)=1/a, a>0, −a/2≦x≦a/2. The noise modified data processes are denoted as y s , y g and y u , respectively. Here, for illustration purposes, the pdfs of these suboptimal SR noises may be found by using the C(x) functions. The same results can be obtained by applying the same approach as in the previous subsection using F 1 (•) and F 0 (•) functions. For the arbitrary symmetrical noise case, we have the condition

p s ( x )= p s ( −x ).  (52)

Therefore, p(y|H 0 ) is also a symmetric function, so that P FA y s =½. By equations (43) and (52), we have

Since p s (x)≧0, we also have

C s ( x 1 )≧ C s ( x 0 ) for any x 1 ≧x 0 ,  (54)

and

C s (0)=½ , C s (−∞)=0, and C s (∞) =1  (55)

From equations (9) and (53), we have the P D of y s , given by

P D y s = ⁢ ∫ - ∞ ∞ ⁢ p 1 ⁡ ( x ) ⁢ C s ⁡ ( x ) ⁢ ⅆ x = ⁢ ∫ - ∞ 0 ⁢ p 1 ⁡ ( x ) ⁢ C s ⁡ ( x ) ⁢ ⅆ x + ∫ 0 ∞ ⁢ ( 1 - C s ⁡ ( - x ) ) ⁢ p 1 ⁡ ( x ) ⁢ ⅆ x = ⁢ ∫ - ∞ 0 ⁢ ( p 1 ⁡ ( x ) - p 1 ⁡ ( - x ) ) ⁢ C s ⁡ ( x ) ⁢ ⅆ x + P D x = ⁢ ∫ - ∞ 0 ⁢ H ⁡ ( x ) ⁢ C s ⁡ ( x ) ⁢ ⅆ x + P D x , ( 56 )

where H(x)≡p 1 (x)−p 1 (−x). FIG. 5 shows a plot of H(x) for several σ 0 values. Finally, from equation (42), we have

=

When A≧μ, since p 1 (−x)≧p 1 (x) when x<0, we have, H(x)<0, x<0. From equation (56), P D y ≧P D x for any H(x), i.e., in this case, the detection performance of this detector cannot be improved by adding symmetric noise. When A<μ and σ 0 ≧σ 1 , then H(x)<0, ∀x<0. Therefore, adding symmetric noise will not improve the detection performance as well. However, when σ 0 ≦σ 1 , H(x) has only a single root x 0 for x<0 and H(x)<0, ∀x<x 0 , H(x)>0, ∀x (x 0 , 0) and detection performance can be improved by adding symmetric SR noise. From equation (56), we have

Furthermore, since γ(−μ; −μ−A, σ 2 0 )=γ(−μ; −μ+A, σ 2 0 ) and γ(−μ; −μ+A, σ 2 0 )≈0 given 2μ−A>>σ 0 , we have x 0 ≈−μ. Therefore,

The pdf of y for the H 1 hypothesis becomes

Hence, when μ is large enough,

=

Note that, as σ 0 decreases P D,opt y s increases, i.e., better detection performance can be achieved by adding the optimal symmetric noise.

Similarly, for the uniform noise case,

Substituting equation (60) for C s (x) in (56) and taking the derivative w.r.t α, we have

Setting it equal to zero and solving, we have a opt =8.4143 in the pdf of uniform noise defined earlier. Additionally, we have P D,opt y s =0.6011.

For the Gaussian case, the optimal WGN level is readily determined since

Let σ 2 2 =σ 2 0 +σ 2 and take the derivative w.r.t σ 2 2 in equation (62), setting it equal to zero and solving, forming

σ 2 2 = 2 ⁢ μ ⁢ ⁢ A ln ⁡ ( μ + A μ - A ) = 8.6562 , ( 63 )

and σ 2 opt =σ 2 2 −σ 2 0 =7.6562, and correspondingly, P D,opt y s =0.5807. Therefore, when σ 2 0 <σ 2 2 , adding WGN with variance σ 2 opt can improve the detection performance to a constant level P yg D,opt .

Table 1 below is a comparison of detection performance for different SR noise enhanced detectors, and shows the values of P D, opt y for the different types of SR noise. Compared to the original data process with P D x =0.5114, the improvement of different detectors are 0.1811, 0.1593, 0.0897 and 0.0693 for optimum SR noise, optimum symmetric noise, optimum uniform noise and optimum Gaussian noise enhanced detectors, respectively.

FIG. 6 shows P D x as well as the maximum achievable P D y with different values of A. The detection performance is significantly improved by adding optimal SR noise. When A≦μ, a certain degree of improvement is also observed by adding suboptimal SR noise. When A is small, x 0 ≈−μ and x 1 ≈μ, the detection performance of the optimum SR noise enhanced detector is close to the optimum symmetric noise enhanced one. However, when A>0.6, the difference is significant. When A>μ=3, H(x)<0; ∀x<0, so that P D,opt y s =P D,opt y u =P D, opt y g =P D x , i.e, the optimal symmetric noise is zero (no SR noise). However, by adding optimal SR noise, P D,opt y is still larger than P D x , i.e., the detection performance can still be improved. When A≧5, the P D improvement is not that significant because P D x >0.97≈1 which is already a very good detector.

The maximum achievable detection performance of different SR noise enhanced detectors with different background noise σ 0 is shown in FIG. 7 . When σ 0 is small, for the optimum SR noise enhanced detectors P D,opt y ≈1, while for the symmetric SR noise case P D,opt y s ≈0.75. When σ 0 increases, P D x increases and the detection performance of SR noise enhanced detectors degrades. When σ 0 ≧σ 1 , p 0 (x) becomes a unimodal noise and the decision function φ is the same as the decision function decided by the optimum LRT test given the false alarm P FA =0.5. Therefore, adding any SR noise will not improve P D . Hence, all the detection results converge to P D x .

FIG. 8 compares the detection performance of different detectors w.r.t. μ when A=1 and σ 0 =1 is fixed. P D x , P D,opt y s and P D,opt y g monotonically decrease when μ increases. Also, there exists a unique μ value μ 0 , such that when μ<μ 0 is small, p 0 is still a unimodal pdf, so that the decision function φ is the optimum one for P FA =0.5. An interesting observation from FIG. 8 is that the P D of the “optimum LRT” after the lowest value is reached, increases when μ increases. The explanation of this phenomenon is that when μ is sufficiently large, the separation of the two peaks of the Gaussian mixtures increases as μ increases so that the detectability is increased. When μ→∞, the two peaks are sufficiently separated, so that the detection performance of “LRT” is equal to the P D when μ=0.

Finally, FIG. 9 shows the ROC curves for the detection problem when N=30 and the different types of i.i.d SR noise determined previously are added. Different degrees of improvement are observed for different SR noise pdfs. The optimum SR detector and the optimum symmetric SR detector performance levels are superior to those of the uniform and Gaussian SR detectors and more closely approximate the LRT curve. For LRT, the performance is nearly perfect (P D ≈1 for all P FA s ).

The present invention thus establishes a mathematical theory for the stochastic resonance (SR) noise modified detection problem, as well as several fundamental theorems on SR in detection theory. The detection performance of a SR noise enhanced detector is analyzed where, for any additive noise, the detection performance in terms of P D and P FA can be obtained by applying the expressions of the present invention. Based on these, the present invention established the conditions of potential improvement of P D via the SR effect, which leads to the sufficient condition for the improvability/non-improvability of most suboptimal detectors.

The present invention also established the exact form of the optimal SR noise pdf. The optimal SR noise is shown to be a proper randomization of no more than two discrete signals. Also, the upper limit of the SR enhanced detection performance is obtained by the present invention. Given the distributions p 1 and p 0 , the present invention provides an approach to determine the optimal SR consisting of the two discrete signals and their corresponding weights. It should be pointed out that the present invention is applicable to a variety of SR detectors, e.g., bistable systems.

The SR detectors that may be implemented with the present invention are shown in FIG. 10 . For example, the nonlinear system block of FIG. 10 can depict the bistable system. Let x=[x 1 , x 2 , . . . , x n ] T be the input to the nonlinear system, and x′=[x 01 , x 02 , . . . , x N ] T be the output of the system as shown, where x′=f(x) is the appropriate nonlinear function. The decision problem based on x′ can be described by decision function φ 0 (•) as shown. It is easy to observe that the corresponding decision function φ(•) for the ‘super’ detector (i.e., the nonlinear system plus detector) is φ(x)=φ 0 (f(x)). Thus, the SR detectors can be viewed as the system in FIG. 10 without the additive SR noise n. To summarize, the present invention admits conventional SR systems and allows improved detection system by adding n as shown in FIG. 10 .

FIG. 11 illustrates a diagram of a SR detection system obtained by a modification of the observed data, x. The statistical properties of the data are changed by adding independent SR noise n to yield a new process y such that y=x+n. This process, in turn is provided as input to the noise modified detector.

Based on the mathematical framework of the present invention, for a particular detection problem, the detection performance of six different detectors are compared, namely, the optimum LRT detector, optimum noise enhanced sign detector, optimum symmetric noise enhanced sign detector, optimum uniform noise enhanced sign detector, optimum Gaussian noise enhanced sign detector and the original sign detector. Compared to the traditional SR approach where the noise type is predetermined, much better detection performance is obtained by adding the proposed optimum SR noise to the observed data process. The present invention thus corresponds with the observed SR phenomenon in signal detection problems, and greatly advances the determination the applicability of SR in signal detection. The present invention can also be applied to many other signal processing problems such as distributed detection and fusion as well as pattern recognition applications.

The present invention may thus be used to increase the probability of detecting signals embedded in non-Gaussian noise. The first step is to record data from an observed data process. Next, stochastic resonance noise is added to said recorded. The appropriate stochastic resonance noise is controlled by determining the stochastic resonance noise probability density function (PDF) that does not increase the detector probability of false alarm.

The SR noise may be determined for the case of a known data probability density function by determining from the known probability density function of the observed data process the stochastic resonance noise PDF that equals λδ(n−n 1 )+(1−λ)δ(n−n 2 ), with values n 1 and n 2 equal to those of the two delta function locations, and with probabilities equal to λ and (1−λ), respectively. More specifically, the stochastic resonance noise PDF may be calculated by determining F i (x)=∫ R N φ(y) p i (y−x)dy i=0,1, using known critical function φ(y) and known data probability density functions p i (•), i=0,1; determining the three unknown quantities n 1 , n 2 , and λ using the known values k 0 , ƒ 01 and ƒ 02 and the following three equations:

ⅆ J ⅆ f 0 ⁢ ( f 01 ⁡ ( k 0 ) ) = ⅆ J ⅆ f 0 ⁢ ( f 02 ⁡ ( k 0 ) ; ( i ) ⅆ J ⅆ f 0 ⁢ ( f 02 ⁡ ( k 0 ) ) = k 0 ; ( ii ) J ⁡ ( f 02 ⁡ ( k 0 ) ) - J ( f 01 ⁡ ( k 0 ) = k 0 ⁡ ( f 02 ⁡ ( k 0 ) - f 01 ⁡ ( k 0 ) ) ; ( iii )

and determining the probability of occurrence for n 1 and n 2 as λ and 1−λ, respectively, using the equation

λ

=

Alternatively, the SR noise for the case of a known data probability density function may be calculated by determining the stochastic resonance noise PDF that consists of a single delta function, δ(n−n 0 ) with value no equal to the delta function location with probability one. The minimum probability of error may be calculated from

P e , min = π 1 ⁡ [ 1 - max f 0 ⁢ G ⁡ ( f 0 , π 0 π 1 ) ]

where G(ƒ 0 , k)=J(ƒ 0 )−kf 0 =P D −kP FA . The single delta function located at n 0 is calculated from n 0 =F 0 −1 (ƒ 0 ), where ƒ 0 is the value the maximizes

.

The SR noise for the case of labeled data with an unknown data PDF may be determined by first calculating the stochastic resonance noise PDF that consists of two delta functions. This step is accomplished by estimating the stochastic resonance noise consisting of two random variables n 1 and n 2 by using many algorithms, such as expectation-maximization (EM) and the Karzen method to estimate the unknown data PDFs, and applying the estimated PDFs and the stochastic resonance noise PDF may be calculated by determining F i (x)=∫ R N φ(y)p i (y−x)dy i=0,1, using known critical function φ(y) and known data probability density functions p i (•), i=0,1; determining the three unknown quantities n 1 , n 2 , and λ using the known value k 0 , and estimated values {circumflex over (ƒ)} 01 , {circumflex over (ƒ)} 02 , and Ĵ in the following three equations:

ⅆ J ^ ⅆ f 0 ⁢ ( f ^ 0 ⁡ ( k 0 ) ) = ⅆ J ^ ⅆ f 0 ⁢ ( f ^ 02 ⁡ ( k 0 ) ; ( i ) ⅆ J ^ ⅆ f 0 ⁢ ( f ^ 02 ⁡ ( k 0 ) ) = k 0 ; ( ii ) J ^ ⁡ ( f ^ 02 ⁡ ( k 0 ) ) - J ^ ( f ^ 01 ⁡ ( k 0 ) = k 0 ⁡ ( f ^ 02 ⁡ ( k 0 ) - f ^ 01 ⁡ ( k 0 ) ) ; ( iii )

and determining the probability of occurrence for n 1 and n 2 as λ and 1−λ, respectively, using the equation

λ

=

The next step is to determine the stochastic resonance noise consisting of two random variables n 1 and n 2 with values equal to those of the two delta function locations and with probabilities equal to those of said stochastic resonance noise probability density function; adding said stochastic resonance noise to said data; applying said fixed detector to the resulting data process.

Finally, a test statistic for signal detection is calculated under a constant probability of false alarm rate (CFAR) constraint, such that the performance of suboptimal, nonlinear, fixed detectors operating in said non-Gaussian noise are improved. Increasing the probability of detecting signals embedded in non-Gaussian noise comprises adding the stochastic resonance noise n 1 and n 2 with probability λ and 1−λ, respectively, to the data, and applying the fixed detector to the resulting data process.

The present invention also provides a method for evaluating functions using f 1 , J(f 0 ), and dJ/df 0 where for any f 0 , the equation x=F 0 −1 (ƒ 0 ) is solved, and the value of f 1 is obtained by

Following is background information relative to the formulas of the present invention and the applicable theorems on convex functions and convex sets.

A. Convex Functions

A function ƒ: I→R is called convex if

ƒ(λ x +(1−λ) y )≦λƒ( x )+(1−λ)ƒ( y )  (64)

for all x,yεI and λ in the open interval (0,1). It is called strictly convex provided that the inequality (64) is strict for x≠y. Similarly, if −ƒ: I→R is convex, then we say that ƒ: I→R is concave.

Theorem A-1: Suppose ƒ″ exists on (a,b). Then ƒ is convex if and only if ƒ″(x)≧0. And if ƒ″(x)>0 on (a,b), then ƒ is strictly convex on the interval.

B. Convex Sets

Let U be a subset of a linear space L. We say that U is convex if x,yεU implies that z=[λx+(1−λ)y]εU for all λε[0,1].

Theorem A-2: A set U ⊂ L is convex if and only if every convex combination of points of U lies in U.

We call the intersection of all convex sets containing a given set U the convex hull of U denoted by H(U).

Theorem A-3: For any U ⊂ L, the convex hull of U consists precisely of all convex combinations of elements of U.

Furthermore, for the convex hull, we have Carathéodory's theorem for convex sets.

Theorem A-4 (Carathéodory's Theorem): If U ⊂ L, and its convex hull of H(U) has dimension m, then for each zεH(U), there exists m+1 points x 0 , x 1 , . . . x m of U such that z is a convex combination of these points.