Article: Function for integral Fourier-like transform
A wavelet is a wave-like oscillation with an amplitude that begins at zero, increases or decreases, and then returns to zero one or more times. Wavelets are termed a "brief oscillation". A taxonomy of wavelets has been established, based on the number and direction of its pulses. Wavelets are imbued with specific properties that make them useful for signal processing.
For example, a wavelet could be created to have a frequency of Middle C and a short duration of roughly one tenth of a second. If this wavelet were to be convolved with a signal created from the recording of a melody, then the resulting signal would be useful for determining when the Middle C note appeared in the song. Mathematically, a wavelet correlates with a signal if a portion of the signal is similar. Correlation is at the core of many practical wavelet applications.
As a mathematical tool, wavelets can be used to extract information from many kinds of data, including audio signals and images. Sets of wavelets are needed to analyze data fully. "Complementary" wavelets decompose a signal without gaps or overlaps so that the decomposition process is mathematically reversible. Thus, sets of complementary wavelets are useful in wavelet-based compression/decompression algorithms, where it is desirable to recover the original information with minimal loss.
In formal terms, this representation is a wavelet series representation of a square-integrable function with respect to either a complete, orthonormal set of basis functions, or an overcomplete set or frame of a vector space, for the Hilbert space of square-integrable functions. This is accomplished through coherent states.
In classical physics, the diffraction phenomenon is described by the Huygens–Fresnel principle that treats each point in a propagating wavefront as a collection of individual spherical wavelets. The characteristic bending pattern is most pronounced when a wave from a coherent source (such as a laser) encounters a slit/aperture that is comparable in size to its wavelength. This is due to the addition, or interference, of different points on the wavefront (or, equivalently, each wavelet) that travel by paths of different lengths to the registering surface. Multiple, closely spaced openings (e.g., a diffraction grating), can result in a complex pattern of varying intensity.
Etymology
The word wavelet has been used for decades in digital signal processing and exploration geophysics. The equivalent French word ondelette meaning "small wave" was used by Morlet and Grossmann in the early 1980s.
Wavelet theory
Wavelet theory is applicable to several subjects. All wavelet transforms may be considered forms of time-frequency representation for continuous-time (analog) signals and so are related to harmonic analysis. Discrete wavelet transform (continuous in time) of a discrete-time (sampled) signal by using discrete-time filterbanks of dyadic (octave band) configuration is a wavelet approximation to that signal. The coefficients of such a filter bank are called the shift and scaling coefficients in wavelets nomenclature. These filterbanks may contain either finite impulse response (FIR) or infinite impulse response (IIR) filters. The wavelets forming a continuous wavelet transform (CWT) are subject to the uncertainty principle of Fourier analysis respective sampling theory: Given a signal with some event in it, one cannot assign simultaneously an exact time and frequency response scale to that event. The product of the uncertainties of time and frequency response scale has a lower bound. Thus, in the scaleogram of a continuous wavelet transform of this signal, such an event marks an entire region in the time-scale plane, instead of just one point. Also, discrete wavelet bases may be considered in the context of other forms of the uncertainty principle.
Wavelet transforms are broadly divided into three classes: continuous, discrete and multiresolution-based.
Continuous wavelet transforms (continuous shift and scale parameters)
In continuous wavelet transforms, a given signal of finite energy is projected on a continuous family of frequency bands (or similar subspaces of the Lp function space L2(R) ). For instance the signal may be represented on every frequency band of the form [f, 2f] for all positive frequencies f > 0. Then, the original signal can be reconstructed by a suitable integration over all the resulting frequency components.
The frequency bands or subspaces (sub-bands) are scaled versions of a subspace at scale 1. This subspace in turn is in most situations generated by the shifts of one generating function ψ in L2(R), the mother wavelet. For the example of the scale one frequency band [1, 2] this function is
with the (normalized) sinc function. That, Meyer's, and two other examples of mother wavelets are:
The subspace of scale a or frequency band [1/a, 2/a] is generated by the functions (sometimes called child wavelets)
where a is positive and defines the scale and b is any real number and defines the shift. The pair (a, b) defines a point in the right halfplane R+ × R.
The projection of a function x onto the subspace of scale a then has the form
with wavelet coefficients
For the analysis of the signal x, one can assemble the wavelet coefficients into a scaleogram of the signal.
See a list of some Continuous wavelets.
Discrete wavelet transforms (discrete shift and scale parameters, continuous in time)
It is computationally impossible to analyze a signal using all wavelet coefficients, so one may wonder if it is sufficient to pick a discrete subset of the upper halfplane to be able to reconstruct a signal from the corresponding wavelet coefficients. One such system is the affine system for some real parameters a > 1, b > 0. The corresponding discrete subset of the halfplane consists of all the points (am, nb am) with m, n in Z. The corresponding child wavelets are now given as
A sufficient condition for the reconstruction of any signal x of finite energy by the formula
is that the functions form an orthonormal basis of L2(R).
Multiresolution based discrete wavelet transforms (continuous in time)
In any discretised wavelet transform, there are only a finite number of wavelet coefficients for each bounded rectangular region in the upper halfplane. Still, each coefficient requires the evaluation of an integral. In special situations this numerical complexity can be avoided if the scaled and shifted wavelets form a multiresolution analysis. This means that there has to exist an auxiliary function, the father wavelet φ in L2(R), and that a is an integer. A typical choice is a = 2 and b = 1. The most famous pair of father and mother wavelets is the Daubechies 4-tap wavelet. Note that not every orthonormal discrete wavelet basis can be associated to a multiresolution analysis; for example, the Journe wavelet admits no multiresolution analysis.
From the mother and father wavelets one constructs the subspaces
The father wavelet keeps the time domain properties, while the mother wavelets keeps the frequency domain properties.
From these it is required that the sequence
forms a multiresolution analysis of L2 and that the subspaces are the orthogonal "differences" of the above sequence, that is, Wm is the orthogonal complement of Vm inside the subspace Vm−1,
In analogy to the sampling theorem one may conclude that the space Vm with sampling distance 2m more or less covers the frequency baseband from 0 to 1/2m-1. As orthogonal complement, Wm roughly covers the band [1/2m−1, 1/2m].
From those inclusions and orthogonality relations, especially , follows the existence of sequences and that satisfy the identities
so that and so that
The second identity of the first pair is a refinement equation for the father wavelet φ. Both pairs of identities form the basis for the algorithm of the fast wavelet transform.
From the multiresolution analysis derives the orthogonal decomposition of the space L2 as
For any signal or function this gives a representation in basis functions of the corresponding subspaces as
where the coefficients are
and Time-causal wavelets
For processing temporal signals in real time, it is essential that the wavelet filters do not access signal values from the future as well as that minimal temporal latencies can be obtained. Time-causal wavelets representations have been developed by Szu et al and Lindeberg, with the latter method also involving a memory-efficient time-recursive implementation.
Mother wavelet
For practical applications, and for efficiency reasons, one prefers continuously differentiable functions with compact support as mother (prototype) wavelet (functions). However, to satisfy analytical requirements (in the continuous WT) and in general for theoretical reasons, one chooses the wavelet functions from a subspace of the space This is the space of Lebesgue measurable functions that are both absolutely integrable and square integrable in the sense that
and
Being in this space ensures that one can formulate the conditions of zero mean and square norm one:
is the condition for zero mean, and
is the condition for square norm one.
For ψ to be a wavelet for the continuous wavelet transform (see there for exact statement), the mother wavelet must satisfy an admissibility criterion (loosely speaking, a kind of half-differentiability) in order to get a stably invertible transform.
For the discrete wavelet transform, one needs at least the condition that the wavelet series is a representation of the identity in the space L2(R). Most constructions of discrete WT make use of the multiresolution analysis, which defines the wavelet by a scaling function. This scaling function itself is a solution to a functional equation.
In most situations it is useful to restrict ψ to be a continuous function with a higher number M of vanishing moments, i.e. for all integer m < M
The mother wavelet is scaled (or dilated) by a factor of a and translated (or shifted) by a factor of b to give (under Morlet's original formulation):
For the continuous WT, the pair (a,b) varies over the full half-plane R+ × R; for the discrete WT this pair varies over a discrete subset of it, which is also called affine group.
These functions are often incorrectly referred to as the basis functions of the (continuous) transform. In fact, as in the continuous Fourier transform, there is no basis in the continuous wavelet transform. Time-frequency interpretation uses a subtly different formulation (after Delprat).
Restriction： when and ,
has a finite time interval
Comparisons with Fourier transform (continuous-time)
The wavelet transform is often compared with the Fourier transform, in which signals are represented as a sum of sinusoids. In fact, the Fourier transform can be viewed as a special case of the continuous wavelet transform with the choice of the mother wavelet
.
The main difference in general is that wavelets are localized in both time and frequency whereas the standard Fourier transform is only localized in frequency. The Short-time Fourier transform (STFT) is similar to the wavelet transform, in that it is also time and frequency localized, but there are issues with the frequency/time resolution trade-off.
In particular, assuming a rectangular window region, one may think of the STFT as a transform with a slightly different kernel
where can often be written as , where and u respectively denote the length and temporal offset of the windowing function. Using Parseval's theorem, one may define the wavelet's energy as
From this, the square of the temporal support of the window offset by time u is given by
and the square of the spectral support of the window acting on a frequency
Multiplication with a rectangular window in the time domain corresponds to convolution with a function in the frequency domain, resulting in spurious ringing artifacts for short/localized temporal windows. With the continuous-time Fourier Transform, and this convolution is with a delta function in Fourier space, resulting in the true Fourier transform of the signal . The window function may be some other apodizing filter, such as a Gaussian. The choice of windowing function will affect the approximation error relative to the true Fourier transform.
A given resolution cell's time-bandwidth product may not be exceeded with the STFT. All STFT basis elements maintain a uniform spectral and temporal support for all temporal shifts or offsets, thereby attaining an equal resolution in time for lower and higher frequencies. The resolution is purely determined by the sampling width.
In contrast, the wavelet transform's multiresolutional properties enables large temporal supports for lower frequencies while maintaining short temporal widths for higher frequencies by the scaling properties of the wavelet transform. This property extends conventional time-frequency analysis into time-scale analysis.
The discrete wavelet transform is less computationally complex, taking O(N) time as compared to O(N log N) for the fast Fourier transform. This computational advantage is not inherent to the transform, but reflects the choice of a logarithmic division of frequency, in contrast to the equally spaced frequency divisions of the FFT (fast Fourier transform) which uses the same basis functions as DFT (Discrete Fourier Transform). It is also important to note that this complexity only applies when the filter size has no relation to the signal size. A wavelet without compact support such as the Shannon wavelet would require O(N2). (For instance, a logarithmic Fourier Transform also exists with O(N) complexity, but the original signal must be sampled logarithmically in time, which is only useful for certain types of signals.)
Definition of a wavelet
A wavelet (or a wavelet family) can be defined in various ways:
Scaling filter
An orthogonal wavelet is entirely defined by the scaling filter – a low-pass finite impulse response (FIR) filter of length 2N and sum 1. In biorthogonal wavelets, separate decomposition and reconstruction filters are defined.
For analysis with orthogonal wavelets the high pass filter is calculated as the quadrature mirror filter of the low pass, and reconstruction filters are the time reverse of the decomposition filters.
Daubechies and Symlet wavelets can be defined by the scaling filter.
Scaling function
Wavelets are defined by the wavelet function ψ(t) (i.e. the mother wavelet) and scaling function φ(t) (also called father wavelet) in the time domain.
The wavelet function is in effect a band-pass filter and scaling that for each level halves its bandwidth. This creates the problem that in order to cover the entire spectrum, an infinite number of levels would be required. The scaling function filters the lowest level of the transform and ensures all the spectrum is covered. See for a detailed explanation.
For a wavelet with compact support, φ(t) can be considered finite in length and is equivalent to the scaling filter g.
Meyer wavelets can be defined by scaling functions
Wavelet function
The wavelet only has a time domain representation as the wavelet function ψ(t).
For instance, Mexican hat wavelets can be defined by a wavelet function. See a list of a few Continuous wavelets.
History
The development of wavelets can be linked to several separate trains of thought, starting with Haar's work in the early 20th century. Later work by Dennis Gabor yielded Gabor atoms (1946), which are constructed similarly to wavelets, and applied to similar purposes.
Notable contributions to wavelet theory since then can be attributed to Zweig’s discovery of the continuous wavelet transform (CWT) in 1975 (originally called the cochlear transform and discovered while studying the reaction of the ear to sound), Pierre Goupillaud, Grossmann and Morlet's formulation of what is now known as the CWT (1982), Jan-Olov Strömberg's early work on discrete wavelets (1983), the Le Gall–Tabatabai (LGT) 5/3-taps non-orthogonal filter bank with linear phase (1988), Ingrid Daubechies' orthogonal wavelets with compact support (1988), Mallat's non-orthogonal multiresolution framework (1989), Ali Akansu's Binomial QMF (1990), Nathalie Delprat's time-frequency interpretation of the CWT (1991), Newland's harmonic wavelet transform (1993), and set partitioning in hierarchical trees (SPIHT) developed by Amir Said with William A. Pearlman in 1996.
The JPEG 2000 standard was developed from 1997 to 2000 by a Joint Photographic Experts Group (JPEG) committee chaired by Touradj Ebrahimi (later the JPEG president). In contrast to the DCT algorithm used by the original JPEG format, JPEG 2000 instead uses discrete wavelet transform (DWT) algorithms. It uses the CDF 9/7 wavelet transform (developed by Ingrid Daubechies in 1992) for its lossy compression algorithm, and the Le Gall–Tabatabai (LGT) 5/3 discrete-time filter bank (developed by Didier Le Gall and Ali J. Tabatabai in 1988) for its lossless compression algorithm. JPEG 2000 technology, which includes the Motion JPEG 2000 extension, was selected as the video coding standard for digital cinema in 2004.
Timeline
First wavelet (Haar Wavelet) by Alfréd Haar (1909)
Since the 1970s: George Zweig, Jean Morlet, Alex Grossmann
Since the 1980s: Yves Meyer, Didier Le Gall, Ali J. Tabatabai, Stéphane Mallat, Ingrid Daubechies, Ronald Coifman, Ali Akansu, Victor Wickerhauser
Since the 1990s: Nathalie Delprat, Newland, Amir Said, William A. Pearlman, Touradj Ebrahimi, JPEG 2000
Wavelet transforms
A wavelet is a mathematical function used to divide a given function or continuous-time signal into different scale components. Usually one can assign a frequency range to each scale component. Each scale component can then be studied with a resolution that matches its scale. A wavelet transform is the representation of a function by wavelets. The wavelets are scaled and translated copies (known as "daughter wavelets") of a finite-length or fast-decaying oscillating waveform (known as the "mother wavelet"). Wavelet transforms have advantages over traditional Fourier transforms for representing functions that have discontinuities and sharp peaks, and for accurately deconstructing and reconstructing finite, non-periodic and/or non-stationary signals.
Wavelet transforms are classified into discrete wavelet transforms (DWTs) and continuous wavelet transforms (CWTs). Note that both DWT and CWT are continuous-time (analog) transforms. They can be used to represent continuous-time (analog) signals. CWTs operate over every possible scale and translation whereas DWTs use a specific subset of scale and translation values or representation grid.
There are a large number of wavelet transforms each suitable for different applications. For a full list see list of wavelet-related transforms but the common ones are listed below:
Continuous wavelet transform (CWT)
Discrete wavelet transform (DWT)
Fast wavelet transform (FWT)
Lifting scheme and generalized lifting scheme
Wavelet packet decomposition (WPD)
Stationary wavelet transform (SWT)
Fractional Fourier transform (FRFT)
Fractional wavelet transform (FRWT)
Generalized transforms
There are a number of generalized transforms of which the wavelet transform is a special case. For example, Yosef Joseph introduced scale into the Heisenberg group, giving rise to a continuous transform space that is a function of time, scale, and frequency. The CWT is a two-dimensional slice through the resulting 3d time-scale-frequency volume.
Another example of a generalized transform is the chirplet transform in which the CWT is also a two dimensional slice through the chirplet transform.
An important application area for generalized transforms involves systems in which high frequency resolution is crucial. For example, darkfield electron optical transforms intermediate between direct and reciprocal space have been widely used in the harmonic analysis of atom clustering, i.e. in the study of crystals and crystal defects. Now that transmission electron microscopes are capable of providing digital images with picometer-scale information on atomic periodicity in nanostructure of all sorts, the range of pattern recognition and strain/metrology applications for intermediate transforms with high frequency resolution (like brushlets and ridgelets) is growing rapidly.
Fractional wavelet transform (FRWT) is a generalization of the classical wavelet transform in the fractional Fourier transform domains. This transform is capable of providing the time- and fractional-domain information simultaneously and representing signals in the time-fractional-frequency plane.
Applications
Generally, an approximation to DWT is used for data compression if a signal is already sampled, and the CWT for signal analysis. Thus, DWT approximation is commonly used in engineering and computer science, and the CWT in scientific research.
Like some other transforms, wavelet transforms can be used to transform data, then encode the transformed data, resulting in effective compression. For example, JPEG 2000 is an image compression standard that uses biorthogonal wavelets. This means that although the frame is overcomplete, it is a tight frame (see types of frames of a vector space), and the same frame functions (except for conjugation in the case of complex wavelets) are used for both analysis and synthesis, i.e., in both the forward and inverse transform. For details see wavelet compression.
A related use is for smoothing/denoising data based on wavelet coefficient thresholding, also called wavelet shrinkage. By adaptively thresholding the wavelet coefficients that correspond to undesired frequency components smoothing and/or denoising operations can be performed.
Wavelet transforms are also starting to be used for communication applications. Wavelet OFDM is the basic modulation scheme used in HD-PLC (a power line communications technology developed by Panasonic), and in one of the optional modes included in the IEEE 1901 standard. Wavelet OFDM can achieve deeper notches than traditional FFT OFDM, and wavelet OFDM does not require a guard interval (which usually represents significant overhead in FFT OFDM systems).
As a representation of a signal
Often, signals can be represented well as a sum of sinusoids. However, consider a non-continuous signal with an abrupt discontinuity; this signal can still be represented as a sum of sinusoids, but requires an infinite number, which is an observation known as Gibbs phenomenon. This, then, requires an infinite number of Fourier coefficients, which is not practical for many applications, such as compression. Wavelets are more useful for describing these signals with discontinuities because of their time-localized behavior (both Fourier and wavelet transforms are frequency-localized, but wavelets have an additional time-localization property). Because of this, many types of signals in practice may be non-sparse in the Fourier domain, but very sparse in the wavelet domain. This is particularly useful in signal reconstruction, especially in the recently popular field of compressed sensing. (Note that the short-time Fourier transform (STFT) is also localized in time and frequency, but there are often problems with the frequency-time resolution trade-off. Wavelets are better signal representations because of multiresolution analysis.)
This motivates why wavelet transforms are now being adopted for a vast number of applications, often replacing the conventional Fourier transform. Many areas of physics have seen this paradigm shift, including molecular dynamics, chaos theory, ab initio calculations, astrophysics, gravitational wave transient data analysis, density-matrix localisation, seismology, optics, turbulence and quantum mechanics. This change has also occurred in image processing, EEG, EMG, ECG analyses, brain rhythms, DNA analysis, protein analysis, climatology, human sexual response analysis, general signal processing, speech recognition, acoustics, vibration signals, computer graphics, multifractal analysis, and sparse coding. In computer vision and image processing, the notion of scale space representation and Gaussian derivative operators is regarded as a canonical multi-scale representation.
Wavelet denoising
Suppose we measure a noisy signal , where represents the signal and represents the noise. Assume has a sparse representation in a certain wavelet basis, and
Let the wavelet transform of be , where is the wavelet transform of the signal component and is the wavelet transform of the noise component.
Most elements in are 0 or close to 0, and
Since is orthogonal, the estimation problem amounts to recovery of a signal in iid Gaussian noise. As is sparse, one method is to apply a Gaussian mixture model for .
Assume a prior , where is the variance of "significant" coefficients and is the variance of "insignificant" coefficients.
Then , is called the shrinkage factor, which depends on the prior variances and . By setting coefficients that fall below a shrinkage threshold to zero, once the inverse transform is applied, an expectedly small amount of signal is lost due to the sparsity assumption. The larger coefficients are expected to primarily represent signal due to sparsity, and statistically very little of the signal, albeit the majority of the noise, is expected to be represented in such lower magnitude coefficients... therefore the zeroing-out operation is expected to remove most of the noise and not much signal. Typically, the above-threshold coefficients are not modified during this process. Some algorithms for wavelet-based denoising may attenuate larger coefficients as well, based on a statistical estimate of the amount of noise expected to be removed by such an attenuation.
At last, apply the inverse wavelet transform to obtain
Multiscale climate network
Agarwal et al. proposed wavelet based advanced linear and nonlinear methods to construct and investigate Climate as complex networks at different timescales. Climate networks constructed using SST datasets at different timescale averred that wavelet based multi-scale analysis of climatic processes holds the promise of better understanding the system dynamics that may be missed when processes are analyzed at one timescale only
List of wavelets Discrete wavelets Beylkin (18) Moore Wavelet
Biorthogonal nearly coiflet (BNC) wavelets
Coiflet (6, 12, 18, 24, 30)
Cohen-Daubechies-Feauveau wavelet (Sometimes referred to as CDF N/P or Daubechies biorthogonal wavelets)
Daubechies wavelet (2, 4, 6, 8, 10, 12, 14, 16, 18, 20, etc.)
Binomial QMF (Also referred to as Daubechies wavelet)
Haar wavelet Mathieu wavelet Legendre wavelet Villasenor wavelet Symlet Continuous wavelets Real-valued Beta wavelet Hermitian wavelet Meyer wavelet Mexican hat wavelet Poisson wavelet Shannon wavelet Spline wavelet Strömberg wavelet Complex-valued
Complex Mexican hat wavelet
fbsp wavelet Morlet wavelet Shannon wavelet Modified Morlet wavelet See also Chirplet transform Curvelet Digital cinema Dimension reduction Filter banks Fourier-related transforms Fractal compression Fractional Fourier transform
Huygens–Fresnel principle (physical wavelets)
JPEG 2000
Least-squares spectral analysis for computing periodicity in any including unevenly spaced data
Multiresolution analysis Noiselet Non-separable wavelet Scale space Scaled correlation Shearlet Short-time Fourier transform Spectrogram
Ultra wideband radio – transmits wavelets
References Further reading
Haar A., Zur Theorie der orthogonalen Funktionensysteme, Mathematische Annalen, 69, pp. 331–371, 1910.
Ingrid Daubechies, Ten Lectures on Wavelets, Society for Industrial and Applied Mathematics, 1992, .
Ali Akansu and Richard Haddad, Multiresolution Signal Decomposition: Transforms, Subbands, Wavelets, Academic Press, 1992, .
P. P. Vaidyanathan, Multirate Systems and Filter Banks, Prentice Hall, 1993, .
Gerald Kaiser, A Friendly Guide to Wavelets, Birkhauser, 1994, .
Mladen Victor Wickerhauser, Adapted Wavelet Analysis From Theory to Software, A K Peters Ltd, 1994, .
Martin Vetterli and Jelena Kovačević, "Wavelets and Subband Coding", Prentice Hall, 1995, .
Barbara Burke Hubbard, "The World According to Wavelets: The Story of a Mathematical Technique in the Making", A K Peters Ltd, 1998, , .
Stéphane Mallat, "A wavelet tour of signal processing", 2nd edition, Academic Press, 1999, .
Donald B. Percival and Andrew T. Walden, Wavelet Methods for Time Series Analysis, Cambridge University Press, 2000, .
Ramazan Gençay, Faruk Selçuk and Brandon Whitcher, An Introduction to Wavelets and Other Filtering Methods in Finance and Economics, Academic Press, 2001, .
Paul S. Addison, The Illustrated Wavelet Transform Handbook, Institute of Physics, 2002, .
B. Boashash, editor, "Time-Frequency Signal Analysis and Processing – A Comprehensive Reference", Elsevier Science, Oxford, 2003, .
Tony F. Chan and Jackie (Jianhong) Shen, Image Processing and Analysis – Variational, PDE, Wavelet, and Stochastic Methods, Society of Applied Mathematics, (2005).
Numerical Recipes: The Art of Scientific Computing http://apps.nrbook.com/empanel/index.html#pg=699 https://web.archive.org/web/20110811154417/http://apps.nrbook.com/empanel/index.html#pg=699 .
How Wavelets Allow Researchers to Transform — and Understand — Data https://www.quantamagazine.org/how-wavelets-allow-researchers-to-transform-and-understand-data-20211013/
External links
1st NJIT Symposium on Wavelets (April 30, 1990) (First Wavelets Conference in USA)
Binomial-QMF Daubechies Wavelets
Wavelets by Gilbert Strang, American Scientist 82 (1994) 250–255. (A very short and excellent introduction)
Course on Wavelets given at UC Santa Barbara, 2004
Wavelets for Kids (PDF file) (Introductory (for very smart kids!))
WITS: Where Is The Starlet? A dictionary of tens of wavelets and wavelet-related terms ending in -let, from activelets to x-lets through bandlets, contourlets, curvelets, noiselets, wedgelets.
The Fractional Spline Wavelet Transform describes a fractional wavelet transform based on fractional b-Splines.
A Panorama on Multiscale Geometric Representations, Intertwining Spatial, Directional and Frequency Selectivity provides a tutorial on two-dimensional oriented wavelets and related geometric multiscale transforms.
Concise Introduction to Wavelets by René Puschinger
A Really Friendly Guide To Wavelets by Clemens Valens
How Wavelets Allow Researchers to Transform — and Understand — Data https://www.quantamagazine.org/how-wavelets-allow-researchers-to-transform-and-understand-data-20211013/ |
Source: WaveletNetworksEduardoMartinMoraud(s0896801)February6,2009Contents1Introduction12FormalDescription32.1Thewavelettransform,atoolfordecomposition................32.2WaveletNetworks.................................43Applications53.1ComputerVision..................................53.2Engineering.....................................64Conclusion7References71IntroductionManyapplicationsofsignalprocessingentaildetecting,extractingandclassifyingspe-cificelementsfromhigh-dimensionaldata.Thesemaybeparticularsoundsfromacousticalsignals,orshapesfromvisualscenes,andmaymostcertainlypresentfeaturesthatdistin-guishthemfromthesurrounding(sushasacolororafrequency).Asearchstrategythatconcentratesontransientswould,underthesecircumstances,enabletoeasilyseparatefea-turesandunderlineedges.Thisis,asamatteroffact,thekeyideaunderlyingthewavelettransform[6].Limitationsofclassicaltools:ClassicaltoolsforsignalprocessingsuchastheFourierTransform(FT)haveinterestingpropertiesforemphasisingfrequentialfeatures.However,becauseofthewaytheyaredefined,theyareunabletodistinguishsignalsthatarestationaryfromotherswhichvaryovertime.ThatisduetothefactthattheFTcorrelatessignalswithsinusoidalwaves(eiωt)whichhavenon-compactsupportandcoverthewholerealline.Thetransformthusyieldsaglobal‘mix’ofinformationthatmakesisdifficulttoaccountforlocalfrequentialproperties.Figures1and2illustratethislimitation.TheFouriertransformsoftwosignalswithidenticalfrequencycomponentsarecompared.Thefirstoneisstationary:f=sin(2πf1·t)+sin(2πf2·t)fort∈[0,5],whereasthesecondistime-varying:g[0,5]=sin(2πf1·t)andg[5,10]=sin(2πf2·t).Itmaybenotedthattheresultingtransformsareidentical,andthatitisthusimpossibleatthesightofthespectrumtodeterminewhatsignalwasatitsorigin.1
00.511.522.533.544.55−2−1012Stationary functionSamplesAmplitude0100200300400500600050100150200250300Fourier TransformFrequencyAmplitudeFigure1:FourierTransformofastationarysig-nalf=sin(2πf1·t)+sin(2πf2·t)fort∈[0,5].Twopeaksatf1andf2areclearlyvisibleinthefrequencydomain.012345678910−2−1012Non−stationary functionSamplesAmplitude0100200300400500600050100150200250300Fourier TransformFrequencyAmplitudeFigure2:FourierTransformofanon-stationarysignalgdefinedas:g[0,5]=sin(2πf1·t)andg[5,10]=sin(2πf2·t).Thetransformisidenticaltothatofthestationarysignal.WhatmakesWaveletsdifferent?Thewavelettransformhasbeenspecificallycon-ceivedforevaluatingtime-varyingfrequencyinformation,anditisespeciallysuitableforhighlightingtime-frequencypropertiesinthatitdecomposesfunctionsovertestfunctionsψs,uthathavecompactsupportandminimalspreadinthetime-frequencydomain.Figure3illustratesthewavelettransformfortheaforementionedsignals,andemphasizesitsutilityfordetectingfrequentialtransients.ThispropertyoftheWTmayalsobeappliedto2Dfunctions–images–inordertodetectedgesandcontrasts.TheexampleinFigure4showsthepossibilitytopointoutcontours,anditconfirmsitsapplicabilityforshapetrackingorfacerecognition. Intheremainderofthisreview,IwillthuspresentthefoundationsofWaveletNetworksandtheirapplicationsinavarietyofdomains.Section2startsbyclearlydefiningtheWavelettransformandtheneuralnetworksthatarederivedfromthem.Section3thenpresentsanoverviewofdifferentapplications,bothforengineeringandcomputervision.Figure3:WaveletTransformoftheprevioussignals:Non-stationary(top)ansstationary(bottom).5010015020025050100150200250300350Figure4:2DWaveletTransformofapicturewith1coefficient.2
2FormalDescription2.1Thewavelettransform,atoolfordecompositionTheWavelettransformisbasedonthesameprinciplesastheFouriertransform,butreliesonafinite-supporttestfunction–thewavelet–whichactsasalocalisedfilteroftheoriginalsignal,makingitpossibletoiterativelyanalysewindowedpartsofthesignalandextractspace-dependentfrequentialinformation.Eachwaveletisderivedfromazero-mean‘mother’functionψ(Fig5)throughtwolineartransformations:(1)Dilatationbyascaleparameter‘s’and(2)translationby‘u’.Theseparametersdeterminethewidthofthewindowandhencedefinetheresolutionofthetransform,asillustratedinFigure6:ψs,u(x)=1√s·ψ(cid:181)x−us(cid:182)Thewavelettransformofafunctionf(x)–foragivenresolution(s,u)–isthengivenas:WTs,u{f(x)}=(cid:90)+∞−∞f(x)·1√sψT(cid:181)x−us(cid:182)dxwhich,intheHilbertspaceL2(R),correspondstotheinnerproduct<f,ψu,s>,i.e.theprojectionofthefunctioninthedirectionofvectorψu,s.Asamatteroffact,theset{ψu,s}u,s∈RrepresentsabasisofL2(R)overwhichanygivensignalwithfiniteenergymaybedecomposedintoitsdifferentfrequencybands:˜f(x)=(cid:88)∀u,s<f,ψu,s>ψu,s(x)(1)ForafinitenumberofcomponentsN,thefunction˜f(x)isanapproximationoftheoriginalfunctioninthespacespannedbytheNvectors{ψs,u(x)}.Interestingly,itmaypossibletoensurethattheapproximationisascloseasdesiredtotheoriginalfunctionbyselectingtheappropriatesizeforthesubspace[6],i.e.∀,∃Nsothat:kf(x)−N(cid:88)n=1<f,ψn>ψn(x)k<Figure5:ExampleofMorletwaveletmotherfunction.Figure6:Waveformsupportintime(‘win-dowing’)andcorrespondingfrequentialresolu-tion(ExtractedfromS.Mallat1999[6])3 Thispropertyautomaticallyleadstotheideaoffilterbanks.Thesecombineparallelde-compositionsoverdifferentresizedwaveletsinanattempttoapproximatenon-linearsignalswhileensuringthatcertaindesiredcomponentsaremaintained.Figure7:ExampleoffilterbankwithNwaveletsappliedtofunctiondecomposition(ExtractedfromWikipedia[1])WaveletNetworksderivelogicallyfromthisstructureandgobeyond,offeringthepos-sibilitytobeappliedforlearningorclassification.Theypresentinterestingcharacteristicsthatwillbedetailedinthefollowingsectionandwhichmakethemespeciallyrelevantforapplicationsincomputervision–suchasfacerecognitionortracking–biologyetc.2.2WaveletNetworksWaveletNetworksattempttocombinethepropertiesoftheWaveletdecompositionpre-viouslydescribed,alongwiththecharacteristicsofneuralnetworks[10].Theirstructurereliesontheaforementionedprinciples–underlyingnon-linearfunctionapproximation–andisgivenbytheequation˜f(x)=(cid:88)iwiψni(x)(2)inwhichtheweightswirepresentthecoefficientsofthenetwork(Fig8).Thesearetobetunedasthenetworklearns,inordertogivepreferencetorelevantcomponentsamongthesetofNwaveletfunctionsΨ=(ψn1,ψn2...ψnN),whereasnon-relevantonesaretobepenalised.Inthisnotation,thevectorniforeachwaveletgathersitscorrespondingparameters,i.e.n=(s,u)inthecaseofthe1DdecompositionoffunctionsinL2(R),orn=(sx,sy,ux,uy,θ)1for2DimagesinL2(R2).Definition:Atthesightofequation(2),thewaveletnetworkiscompletelydefinedbythetuple(Ψ,w).Itsoptimisedcomponentsmaybeobtainedbycalculatingtheweightswiandwaveletparametersnthatminimisetheleast-squareerrorfunctionforf(x),i.e.theonesthatmakethemodelfitbettertotheoriginalfunctionf:minw,nkf(x)−N(cid:88)i=1wiψni(x)k2ThiscalculationhenceimpliesfindingthemostsuitableNwaveletsonwhichtoproject,alongwiththeweightthateachcomponentoughttobegiven–howmuchitoughttocon-tributetotheoveralldescriptionoff(x)–inordertomaximisetheapproximation˜f(x).Suchprocedureistobecarriedoutduringthelearningphasesoastoadaptthenetworktothesetoftrainingdatapoints{Tn}.Letusremindthatminimisingtheleast-squareerrorfunctioninL2(R2)correspondstofindingthefunction˜f(x)thatreducestheeuclideandistancetoeachpointinthetrainingdata,i.e.theonethatminimisesthelostenergythatisduetotheapproximation.1θrepresentstheorientationinthe2Dplane4 Figure8:WaveletNetworkstructure(similartotheperceptron).Examplewithmultipleinputsandmultipleoutputs(ExtractedfromChen2006[2])Figure9:(Left)Initialdistributionofthewavelets(4×4)and(Right)theirpositionsafteroptimisation(ExtractedfromV.Kruger1999[5])WeightDistribution:Itmaybenotedfromequations(1)and(2)thatthereisatightlinkbetweentheweightsofthenetworkwiandthewaveletdecomposition<f,ψni>.Asamatteroffact,thevaluesforwiareautomaticallyprovidedbythewavelettransform[10].Asaresult,thefinaldistributionofweights(afteroptimisation)inheritsthesuitabilityofwaveletsfor‘featuredetection’.Thewiarethusautomaticallytunedsoastoprioritizeprojectionsthathighlighttransients,whereaswaveletsencodinglessrelevantpartsofimagesarepenalised.Theresultisanetworkwhichisdirectlyrelatedtotheunderlyingimagestructure[5],andinwhichallparameters(wi,n)arejointlyfittedfromdata.ThischaracteristicisclearlyillustratedinFig9(rightimage)wherethe16waveletshavebeendistributedatlocationswherethebiggesttransientsmaybeencountered,suchasforinstanceabovetheeyebrows,atthelimitofthehairoraroundthemouth.Algorithm:Theinitialisationofthenetworkmayconsistofanarbitrarilydistributedsetofwavelets,i.e.homogeneouslydistributedtestfunctionswithrandomorientationsθandconstantdilationsandscalingfactors.Nevertheless,sincewaveletsarerapidly-vanishingfunctions,theparametersmayneedtobeconstrainedinordertopreventdegeneratedwaveletshapes[3].Asfortheupdatingrequirements,backpropagationalgorithmsarecommonlyemployedtotunetheparameters.Thistechniqueisbroadlyusedforteachingfeed-forwardneuralnetworks,andisthusperfectlyadaptedtoourwavelet-basedcase.Inaddition,itpresentsmanyadvantagessuchsuchasquickconvergencetime.Likewise,itmaybepossibletomakeuseofoptimisationalgorithmssuchastheLevenberg-Marquardtmethod,whichisespeciallysuitableforleast-squareproblems.3ApplicationsThepropertiesthathavebeenemphasiseduntilnowmakeWaveletNetworksespeciallyinterestingforawiderangeofapplicationsinengineering,computerscienceorbiology[9].Thesemayrangefromclassification,tofeatureextractionorapproximationofcomplexnon-linearfunctions.Somerelevantexampleswillbefurtherdiscussedinthefollowinglines,inordertogiveaflavourofthepossibilitiesofferedbythistechnique.3.1ComputerVisionManyprobabilisticapproacheshavebeendevelopedforcomputervision,suchasneuralnetworks,PCAorEigenfaces.Thesemethodstendtolearnthevarianceofgrey-valuepixels5 overasetoftrainingdata,andthenmakeuseofthatpixel-basisknowledgetoclassifynewimages.Thisisperformedindependentlyoftheobjectitself.TheinterestofWaveletNetworksreliesontheirabilitytobedirectlyrelatedtotheunderlyingstructureoftheimage[5].Thewaveletsfunctions–onwhichthemodelisbuilt–are‘natural’featuredetectors[7]andareindependent,forinstance,ofilluminationchanges.Figure10pointsouthowwaveletsdo,forinstance,automaticallyadopttheorientationθoftheimage.Moreover,theyprovidearesolutionthatmaybetunedinordertoconcentrateongivenregions,makingthemspeciallysuitableforsurveillanceortrackingapplications.AnexampleforfacetrackingisdescribedbyKruegeret.al.[5]inwhich16sets(4×4)of4Gaborwavelets(withcorrespondingorientations0,π/4,π/2and3π/4)aredistributedwithintheregionofinterest(theface).Theresultingprojectionsarethencalculatedusinganeuralnetworkandthesolutionsofeachfilterresponseareobtained.Inordertobeabletotracktheface,anupdateisperformedforeachframe,whichimplies(1)are-parametrisationofthemodel(soastofollowthemovementofthetarget)and(2)anoptimisationofthepreviousweightstoaccountforchangesintheimage.Thisleadstothetrackingshowninfigure11.Figure10:Differentwaveletssharingtheori-entationofthefeatureintheimage(left).Bestonesautomaticallyselected(right)(extractedfromKrueger2002[5])Figure11:ExamplesofFacetracking(ex-tractedfromKrueger2002[5])3.2EngineeringRobotcontrol:Robotmotionisdescribedbycomplexnon-lineardynamicsequationswhichincludetime-dependentparametersandsystemuncertainties,asobservedinthedif-ferentialdescription:u=M(q)¨q+C(q,˙q)˙q+g(q)Inthisframework,approximationsthroughnon-linearnetworksturnouttobeveryusefulforlearningcontrolpatterns[4],solvinginversekinematicsproblems[2]andsynthesisingcorrectbehavior.Thispurposehasbeenpreviouslystudiedandsolvedbymeansofneuralnetworksbasedonradialbasisfunctions(RBF),i.e.functionsthatdependonlyonthedistancetoareferencepoint(orcentre)ci:˜f(x)=N(cid:88)i=1wiφ(kx−cik)However,foragivenfunction,theRBFnetworkmaynotbeunique,norparticularlyefficient.ThemodeldevelopedbyKaticet.al.[4],forexample,replacesthisactivationfunctionbyawavelet-basednetworkwhichthenplaystheroleofarobustcontroller,helpscompensateuncertaintieswhenthesystemisincontactwiththeenvironment,andyieldsmuchmorecomputationalefficientresults.Promisingresultshavebeenachievedinthiscontextforcontrollingmanipulatorrobotssuchastheoneinfigure12.6 Productqualitymonitorisation:Thomaset.al.[8]extendstheconceptofwaveletnetworkstocopewiththerequirementsofproductionlines.Thispurposeentailsmonitoringlargenumberofnon-stationarysignalswhichareobtainedformsensors,andperformingfeatureextractionandclassificationsoastocomeupwithadiagnosissystem.Inthisenvironment,thewaveletnetworkconcentratesontransientsandisusedtoextractspecificfeaturesfromthesignalsandtorecognisethestateofthesystem.Thismodelmakesuseofdifferentwaveletnetworks,oneforeachfeaturegroup,asillustratedinFigure13,andincludesageometry-basedcriterionformodelselection.Successfulresultshavebeenachievedforengineknockdetection.Thisasserts,fromamoregeneralperspective,theinterestofwaveletnetworksfordealingwithindustrialproblemswhichinvolvenoisysensorsandnon-stationaryreadings.Figure12:ManipulatorPUMA560forwhichInversekinematicsmaybecalculatedthroughwaveletnetworks(ExtractedfromChen2006[2])Figure13:Featureextractionphasewithsev-eralwaveletnetworks(ExtractedfromThomas1996[8])4ConclusionThisreviewhaspresentedtheWavelettransformanditsusewithinfeed-forwardneuralnetworks.Themathematicalfoundationshaveemphasizedimportantpropertiesregardingfeaturedetectionandnon-linealfunctionapproximation,whichhavebeenshowntobeofgreatinterestforapplicationsinengineering,roboticsandcomputervision.Examplesoffacetrackingordynamiccontrolhavebeenhighlightedasproofofwhatthistechniquemayleadtointhefuture.Still,asstatedin[6],‘theworldoftransientsisconsiderablylargerandmorecomplexthanthegardenofstationarysignals’,andthisreviewdoescertainlynotattempttogiveafulloverviewofallthetechniquescurrentlyavailableinthisfield.Ithopes,nevertheless,tohaveprovidedaclearbackgroundofthedifferencewithotherapproaches,andtomotivatethereadertocontinuedeepeninginthepossibilitiesofferedbythisdomain.References[1]Bankfilters,2009.http://en.wikipedia.org/wiki/Wavelet(lastmodified:January30,2009).[2]H.Cheng,W.shanChen,andT.Xie.Waveletnetworksolutionfortheinversekine-maticsprobleminroboticsmanipulator.JournalofZhejiangUniversitySCIENCEA,4(7):525–529,2006.[3]J.Daugman.Uncertaintyrelationforresolutioninspace,spatialfrequency,andorien-tationoptimizedbytwo-dimensionalvisualcorticalfilters.JournaloftheopticalsocietyofAmerica,pages1160–1169,1985.7 [4]D.KaticandM.Vukobratovic.Waveletneuralnetworkapproachforcontrolofnon-contactandcontactrobotictasks.IntelligentControl,1997.Proceedingsofthe1997IEEEInternationalSymposiumon,pages245–250,Jul1997.[5]V.KruegerandG.Sommer.Gaborwaveletnetworksforefficientheadposeestimation.ImageandVisioncomputing,20:665–672,2002.[6]S.Mallat.AWaveletTourofSignalProcessing.AcademicPress,1999.[7]B.ManjunathandramaChallapa.Aunifiedapproachtoboundaryperception:Edges,texturesandillusorycontours.InIEEETransactionsonNeuralNetworks,volume1,pages96–108,Jan1993.[8]J.-H.ThomasandB.Dubuisson.Adiagnosticmethodusingwaveletsnetworksap-plicationtoengineknockdetection.Systems,Man,andCybernetics,1996.,IEEEInternationalConferenceon,1:244–249vol.1,Oct1996.[9]M.Thuillard.Areviewofwaveletnetworks,wavenets,fuzzywavenetsandtheirappli-cations.AdvancesinComputationalIntelligenceandLearning,pages43–60,2002.[10]Q.ZhangandA.Benveniste.Waveletnetworks.InIEEETransactionsonneuralnetworks,volume3,pages889–898,1992.8 |