Time-Frequency Analysis

When we transform a non-stationary (or time-varying) signal into the frequency domain using a Fourier transform, we throw away almost all of the information about time. If this information is important, then what we want is a signal representation that includes both frequency and time. This can be accomplished with either time-frequency (Wigner, 1932; Cohen, 1989; Hlawatsch and Boudreaux-Bartels, 1992; Cohen, 1995) or wavelet transforms (Rioul and Vetterli, 1991).

The Fourier transform is a representation of a signal as a weighted sum of sinusoids of different frequencies. It provides information about the spectral content of the signal, but not when any particular component has occurred. This is a critical limitation of this type of spectral analysis. For example, the spectrum of someone speaking might allow one to determine some of the speaker's characteristics (high or low pitched voice, and therefore perhaps their sex), but certainly wouldn't provide enough information to understand what they were saying. One common approach to determining temporal information is the short-time Fourier transform, or spectrogram. To compute a spectrogram, we look at a narrow window of the signal and compute the Fourier transform for that short duration. We then shift the window slightly in time, transform again, and repeat. The result is a sequence of spectra along time, with our uncertainty of "when" reduced from the entire signal duration to the window duration. Unfortunately, we cannot increase our temporal resolution without limit using this method.

The time-bandwidth product theorem for the Fourier transform states that, for a short duration signal, the transform will have a wide bandwidth, independent of the signal's actual spectral content. In other words, the frequency domain representation will have low resolution. Conversely, a long duration signal will have a narrow bandwidth (high frequency resolution). This is a consequence of the fact that the Fourier transform is defined for signals of infinite length (ideal stationarity), and that we "fudge" it for finite signals by assuming they are periodic and infinite. Thus, there is a fundamental tradeoff in this method for computing the spectrogram: frequency resolution versus temporal resolution (note that this has absolutely nothing to do with Heisenberg's uncertainty principle, or in fact any physical law).

Time-frequency analysis methods are based on joint distribution representations of signals, rather than separate time and frequency representation pairs. This is analogous to joint distributions for a pair of random variables, which includes information not only about their individual statistical properties, but also their correlations. Rather than use short windows of a signal to compute spectral information for short periods of time, the entire signal is used to determine the spectral content at any given point in time. There are a number of approaches to computing such distributions, and a formalism has been developed for expressing any time-frequency distribution, including the Fourier transform (Cohen, 1966).

The following graph displays an example time-frequency transform for a speech signal (in this case, the Japanese word "kaze"). The top graph shows the signal along time; the bottom the signal's frequency components along time.

The wavelet transform is an alternative method for spectral analysis of nonstationary signals. It represents a signal as a set of basis functions called wavelets. Unlike the Fourier transform, whose basis functions (sinusoids) are of infinite duration, wavelet functions have (for all practical purposes) finite extent. Each wavelet acts like a bandpass filter of finite duration, and thus a set of them can break a signal into frequency components at particular intervals of time. In practice, the individual wavelets are derived from a single prototype by scaling (producing variation in frequency) and shifting (providing different temporal windows). The wavelet functions are defined such that, as they are scaled to pass different frequencies, their duration also changes. High frequency wavelets have short duration, and low frequency ones have longer duration. Because of this scaling operation, the wavelet transform falls into the category of time-scale transformations (Cohen, 1995).

about this page

This is an archive copy of the occasional S-Squared Newsletter. The current issue can be reached by clinking here, or on the "news" link at the top of the page.