UK Nonlinear News,
Sept. 1995
Introducing Higher Order Statistics (HOS)
Introducing Higher Order Statistics (HOS) for the
Detection of
Nonlinearities
- Introduction
- What are HOS ?
- Why use HOS ?
- Frequency domain methods for detecting nonlinearities
- Conclusions
- Getting more information
- Some HOS jargon
- References
Introduction
Engineering judgement concerning the predictability of a signal is
often based on an examination of the signal spectrum. The conclusion
is then drawn that if a signal has a flat or near-to-flat spectral
density that the quality of the prediction will be poor. While this
line of reasoning can provide useful guidelines in the design of
linear predictive systems it is not true in general not least because it ignores
the existence of purely deterministic mechanisms which generate signals with flat or
near-to-flat spectral densities.
Speech or music signals are generated mechanically by systems with
nonlinear dynamics. If the prediction and coding quality of such
signals is to be improved then more of the information available in
the signal must be used : the signal higher order statistics (HOS)
must be exploited.
The aim of this article is to introduce HOS to a wide audience
(assuming basic knowledge of signal processing and statistics), and to
outline some of the reasons why they can be useful in practical
applications.
History of Higher Order Statistics (HOS)
Several key papers in HOS were published in the 1960's, but most of
these papers took a statistical and theoretical viewpoint of the
subject. It was not until the 1970's the people started to apply HOS
techniques to real signal processing problems. The last 15 years has
seen a revival of interest in HOS techniques, and there is now a
growing number of researchers around the world working in this field.
In recent years the field of HOS has continued its expansion, and
applications have been found in fields as diverse as economics,
speech, seismic data processing, plasma physics and optics. Many
signal processing conferences (ICASSP, EUSIPCO) now have sessions
specifically for HOS, and an IEEE Signal Processing Workshop on HOS
has been held every two years since 1989 (the most recent one took
place in June 1995 in Spain).
[contents]
What are HOS ?
HOS measures are extensions of second-order measures (such as the
autocorrelation function and power spectrum) to higher orders. The
second-order measures work fine if the signal has a Gaussian (Normal)
probability density function, but as mentioned above, many real-life
signals are non-Gaussian.
The easiest way to introduce the HOS measures is just to show some
definitions, so that the reader can see how they are related to the
familiar second-order measures.
Here are definitions
for the time-domain and frequency-domain third-order HOS measures,
assuming a zero-mean, discrete signal x(n),
Time domain measures
In the time domain, the
second order measure is the autocorrelation function
R(m) = <x(n) x(n+m)>
where <> is the expectation operator.
The third-order
measure is called the third-order moment
M(m1,m2) = <x(n) x(n+m1) x(n+m2)>
Note that the third-order moment depends on two independent
lags m1 and m2. Higher order moments can be
formed in a similar way by adding lag terms to the above equation.
The signal cumulants can be easily derived
from the moments.
Frequency domain
In the frequency domain the second-order measure is called the power
spectrum P(k), and it can be calculated in two ways:
- Take a Discrete Fourier Transform (DFT) of the autocorrelation
function ;
P(k) = DFT[ R(m) ].
- Or: Multiply together the signal Fourier Transform
X(k)
with its complex conjugate ;
P(k) = X(k) X*(k)
At third-order the bispectrum B(k,l)can be calculated in
a similar way:
- Take a Double Discrete Fourier Transform (DDFT) of the
third-order cumulant ;
B(k,l) = DDFT[ M(m1,m2) ]
- Or: Form a product of Fourier Transforms at different
frequencies ;
B(k,l) = X(k) X(l) X*(k+l)
More will be said about frequency-domain measures below
Now just as the second-order measures are related to the signal
variance so the third-order measures (third-order cumulant and
bispectrum) are related to the signal skewness, the
fourth-order measures (fourth-order cumulant and trispectrum) are
related to the signal kurtosis, and higher-order measures are
related to higher-order moments of the signal.
[contents]
Why use HOS ?
Some of the key advantages to such techniques over traditional
second-order techniques are
- Second-order measures (such as
the power spectrum and autocorrelation functions) contain no phase
information. As a consequence of this
- non-minimum phase
signals cannot be correctly identified by 2nd-order
techniques. This is important in the field of linear processes, which
will not be discussed in this article.
- certain types of phase coupling (associated with
nonlinearities)
cannot be correctly identified using 2nd order techniques.
- Any Gaussian signal is completely characterised by its mean and
variance. Consequently the HOS of Gaussian signals are either zero (e.g. the
3rd order moment of a Gaussian signal is zero), or contain redundant
information. Many signals encountered in practice have non-zero HOS,
and many measurement noises are Gaussian, and so in principle the HOS
are less affected by Gaussian background noise than the 2nd order
measures. (e.g. the power spectrum of a deterministic signal
plus Gaussian noise is very different from the power spectrum
of the signal alone. However the bispectrum of the signal + noise is,
at least in principle, the same as that of the signal).
[contents]
Frequency domain methods for detecting nonlinearities
Nonparametric frequency domain methods (such as the power spectrum)
remain popular in fields where signals are of an unkown type, as a
rough-and-ready tool for investigation. In the field of HOS there are
extensions to the familiar power spectrum at 3rd, 4th and higher
orders. These "Higher Order Spectra", or "Polyspectra" provide supplementary information to
the power spectrum. The 3rd order polyspectrum is the easiest to
compute, and hence the most popular, and is called the
bispectrum. The 4th order polyspectrum is called the
trispectrum. These can be estimated in a way similar
to the power spectrum, but more data is usually needed to get reliable
estimates. For this reason it is often not practicable to compute the
polspectra of high orders.
Closely related to the bispectrum is the 3rd-order coherence measure, the
bicoherence.
An example of a bicoherence
plot is shown in the Figure below.
Figure 1 - an example of a bicoherence magnitude plot of a male
vowel sound. The horizontal and vertical scales are frequency scales
k and l respectively, in kHz.
It is evident that the bicoherence has two independent frequency axes
k and l, and that the bicoherence magnitude emerges from
the frequency plane. We only consider the bicoherence in a small
region of the k,l plane because of symmetry relations. The
bicoherence has phase too, and under certain circumstances that needs
to be examined too.
The interpretation of the bicoherence varies depending on the type of
signal under study.
- For stochastic wide-band signals, the level of the bicoherence
gives a measure of the signal skewness (a statistical
test for skewness based on the bispectrum was devised by Hinich in 1982).
- For deterministic signals, a peak in the bicoherence indicates
the presence of Quadratic Phase Coupling (QPC), which
is a specific type of nonlinearity.
The areas of interest in this bicoherence plot are annotated below
Figure 2 - annotated bicoherence plot.
A peak in the bicoherence magnitude of this speech signal at frequnecy
pair (k,l) indicates
QPC between frequency components at the frequencies k, l
and k+l.
We observe that there is significant bicoherence magnitude in several
regions, at the frequency pairs (in kHz) (1.0,0.5), (2.5, 0.5), (3.5,
0.5). These areas of high bicoherence magnitude are indications of
QPC between frequency components at the triplets (1,0,0.5,1.5)
(2.5,0.5,3.0) and (3.5, 0.5, 4.0). In this case it would probably be
necessary to also look at the phase of the bicoherence as well in
order to make sure that these peaks did indeed indicate coupling
(under some circumstances bicoherence magnitude peaks are necessary
but insufficient evidence of QPC). There is very little bicoherence magnitude
content at frequencies above the diagonal line k+l=4.5kHz, this
is due to the fact that this signal has low energy above
4.5kHz.
[contents]
Conclusions
We have attempted to give a brief overview of some of the ideas behind
the use of HOS in signal processing. Key to these ideas is the fact
that many signals in real life cannot be adequately modelled using the
traditional second-order measures such as the power spectrum and
autocorrelation function. Much has already been achieved, but there
is still a great deal of work to do before HOS measures become as
familiar and comprehensible as their second-order counterparts.
Getting More Information
Further information about HOS can be obtained by visiting the
HOS home
page http://www.ee.ed.ac.uk/~hos/. This page contains links to research groups around
the world with interests in HOS, as well as hos bibliographies and a
noticeboard of forthcoming HOS events.
In Edinburgh, our own special areas of interest include HOS in Speech Signals, HOS
for seismic deconvolution, Chaos in Speech, Nonlinear signal
prediction. For more information on these areas, or general questions
concerning this article, please contact
- Steve McLaughlin (any area of
nonlinear signal processing).
- Achilleas Stogioglou (seismic
deconvolution using parametric HOS methods),
- Justin Fackrell (detecting
nonlinearities in speech using HOS).
[contents]
Some HOS jargon
- moments
Moments are statistical measures which characterise signal properties.
We are used to using the mean and the variance (the first and second
moments, respectively) to characterise a signal's probability
distribution, but unless the signal is Gaussian (Normal) then moments
of higher orders are needed to fully describe the distribution. In
practice in HOS we usually use the cumulants rather than the moments
(see below).
- cumulants
The nth order cumulant is a function of the moments of orders
up to (and including) n. For reasons of mathematical
convenience, HOS equations/discussions most often deal with a signal's
cumulants rather than the signal's moments.
- polyspectra
This term is used to describe the family of all frequency-domain spectra, including the
2nd order. Most HOS work on polyspectra focusses attention on the bispectrum
(third-order polyspectrum) and the trispectrum (fourth-order polyspectrum).
- bicoherence
This is used to denote a normalised version of the bispectrum. The
bicoherence takes values bounded between 0 and 1, which make it a
convenient measure for quantifying the extent of phase coupling in a
signal. The normalisation arises because of variance problems of the
bispectral estimators for which there is insufficient space to
explain (see Hinich 1982
for more details).
[contents]
References
The number of papers published on HOS continues to rise each year.
One of the most widely referenced papers is the tutorial paper by
Mendel (referenced below). Below we also reference the first textbook
on HOS, which was published in 1993.
J M Mendel
Tutorial on Higher Order Statistics (Spectra) in signal processing and
system theory: theoretical results and some applications
Proceedings of the IEEE, 79(3),
pp 278-305, March, 1991.
C L Nikias and A P Petropulu
Higher-Order Spectra analysis
PTR Prentice Hall, New Jersey, 1993.
M J Hinich
Testing for Gaussianity and linearity of a stationary time series
Journal of Time Series Analysis,3(3),
pp 169-176, 1982.
[contents]
<< Move to UK Nonlinear News
Issue Two index page
uk-nonl@ucl.ac.uk
15 Sept. 1995