Discrete Wavelet Transforms
Abdourrahmane M. ATTO Polytech Annecy-Chambery - University of Savoie E-mail :
[email protected] - Tel. : 04 50 09 65 27
Course notes - IAI Draft - February 8, 2015
CONTENTS
CONTENTS
Contents 1 Introduction
3
2 1D Discrete wavelet packet decomposition
2.1 2.2 2.3 2.4
4
Splitting lemma on functional subspaces of L (R) . . . . . . . . . Scaling function, wavelets, wavelet lters . . . . . . . . . . . . . . Discrete Wavelet Transform (DWT): signal decomposition examples DWT and frequencies: limit case of the Shannon wavelet packets . . 2
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
3 2-D separable wavelets and wavelet packets
3.1 3.2 3.3 3.4
13
Separability . . . . . . . . . . . . . . . . . . . . . . . . . . 2D DWT and DWPT input space and trees . . . . . . . . . . Shannon DWT and DWPT trees . . . . . . . . . . . . . . . 2D Discrete Wavelet Transform: image decomposition example
A. Atto - Polytech Annecy-Chambry
4 9 11 12
. . . .
p. 2
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
13 14 15 17
1 INTRODUCTION
1
Introduction
W
AVELET transforms have become alternatives to Fourier transforms in many signal and image processing applications. This course provides a short introduction to 1D and 2D Discrete Wavelet and
Wavelet Packet Transforms. More details can be found in [1], [2]. In what follows, IFf stands for the Fourier transform of f ∈ L (R) ∪ L (R), with IFf(ω) = 1
f ∈ L (R). For f ∈ L (R ) ∪ L (R ), its Fourier transform is given by Z IF f(ω1 , ω2 ) = f(t1 , t2 )e−i(ω1 t1 +ω2 t2 ) dt1 dt2 . 1
1
2
2
2
Z
f(t)e−iωt dt if
2
R
R2
Modulus of 2D FFT
Image Lena
A. Atto - Polytech Annecy-Chambry
p. 3
2 1D DISCRETE WAVELET PACKET DECOMPOSITION
2
1D Discrete wavelet packet decomposition
2.1
Splitting lemma on functional subspaces of L2 (R)
Let Φ be a function such that {τk Φ : k ∈ Z} is an orthonormal system of L2 (R), where τk Φ : t 7−→ Φ(t − k). Let U be the closure of the space spanned by this orthonormal system. Consider two Quadrature Mirror Filters (QMF) H0 and H1 , where the QMF condition stands for the unitarity (conjugate transpose is the inverse) of the matrix M(ω) =
for every ω. Assume that
H0 (ω) H1 (ω) H0 (ω + π) H1 (ω + π)
H0 (ω) =
√1 2
H1 (ω) =
√1 2
P `∈Z
P `∈Z
(1)
,
h0 [`]e−i`ω ,
(2) h1 [`]e
−i`ω
√
where h0 and h1 pertain to `2 (Z). The functions H0 and H1 are, up to the factor 1/ 2, the Fourier transforms of h0 and h1 , respectively. We also assume that the functions H0 and H1 are such that H1 (ω) = e−iω H0 (ω + π).
(3)
where z stands for the complex conjugate of z. The QMF condition then reduces to H20 (ω) + H20 (ω + π) = 1.
A. Atto - Polytech Annecy-Chambry
p. 4
2 1D DISCRETE WAVELET PACKET DECOMPOSITION 2.1 Splitting lemma on functional subspaces of L2 (R)
De ne a sequence {Wj,n,k : k ∈ Z} of elements of L2 (R) by applying the following operations: First, set (initialization)
√ P W0 (t) = 2 `∈Z h0 [`]Φ(2t − `)
√ P W1 (t) = 2 `∈Z h1 [`]Φ(2t − `).
Then, apply recursively, the following operations, for all n ≥ 1, √ P W2n (t) = 2 `∈Z h0 [`]Wn (2t − `)
√ P W2n+1 (t) = 2 `∈Z h1 [`]Wn (2t − `).
(4)
(5)
Finally, apply scaling operators Wj,n (t) = 2−j/2 Wn (2−j t)
(6)
for j ≥ 1 and translation operators Wj,n,k = τ2j k Wj,n , Wj,n,k (t) = 2−j/2 Wn (2−j t − k)
(7)
for k ∈ Z. Consequence: The set {Wj,n,k : k ∈ Z} is orthonormal. The decomposition of the function space U obtained by using the above recursive operations is called splitting
lemma .
A. Atto - Polytech Annecy-Chambry
p. 5
2 1D DISCRETE WAVELET PACKET DECOMPOSITION 2.1 Splitting lemma on functional subspaces of L2 (R)
This splitting lemma can be expressed in terms of functional subspaces U = W1,0 ⊕ W1,1 ,
(8)
Wj,n = Wj+1,2n ⊕ Wj+1,2n+1 ,
(9)
and
for every j = 1, 2, · · · , and every n ∈ Ij with (10)
Ij = {0, 1, · · · , 2j − 1}.
The sets {Wj+1,2n,k : k ∈ Z} and {Wj+1,2n+1,k : k ∈ Z} are orthonormal bases of the vector spaces Wj+1,2n and Wj+1,2n+1 , respectively. The tree of gure 1 illustrates such a decomposition. U W1,0 W2,0
W1,1 W2,1
W2,2
W2,3
W3,0 W3,1 W3,2 W3,3 W3,4 W3,5 W3,6 W3,7
Figure 1: Decomposition of U down to level j = 3. This tree is a multiscale/multiresolution analysis du to scaling factors in 2j .
A. Atto - Polytech Annecy-Chambry
p. 6
2 1D DISCRETE WAVELET PACKET DECOMPOSITION 2.1 Splitting lemma on functional subspaces of L2 (R)
The projection of a function f =
P `∈Z
c[`]τk Φ ∈ U on Wj,n yields the coecients Z cj,n [k] = f(t)Wj,n,k (t)dt,
(11)
R
with c0,0 = c. According to the splitting lemma (9), Wj,n is decomposed into two packets Wj+1,2n and Wj+1,2n+1 . For ∈ {0, 1}, the coecients of f on the packet Wj+1,2n+ are
Z
cj+1,2n+ [k] =
(12)
f(t)Wj+1,2n+,k (t)dt. R
We have (in quadratic or pointwise sense, depending on the lter) Wj+1,2n+,k =
X
(13)
h [` − 2k]Wj,n,` .
`∈Z
Indeed: Remark 1 According to (5), (6) and (7), and for ∈ {0, 1}, we have Wj+1,2n+,k (t) = τ2j+1 k Wj+1,2n+ (t) = τ2j+1 k 2−(j+1)/2 W2n+ (2−(j+1) t) = X √ X h [`]Wn (2−j t − `) = τ2j+1 k 2−j/2 h [`]Wn (2−j t − `) = τ2j+1 k 2−(j+1)/2 2 −j/2
2
X
`∈Z −j
−j/2
h [`]Wn (2 t − 2k − `) = 2
`∈Z
X `∈Z
`∈Z −j
h [` − 2k]Wn (2 t − `) =
X
h [` − 2k]Wj,n,` (t).
`∈Z
The equalities above must be understood in the quadratic sense. If the impulse response of the lters have nite length, these equalities hold true pointwise. A. Atto - Polytech Annecy-Chambry
p. 7
2 1D DISCRETE WAVELET PACKET DECOMPOSITION 2.1 Splitting lemma on functional subspaces of L2 (R)
From Eqs. 12 and 13, we can write, with convergence in `2 (Z), cj+1,2n+ [k] =
X
(14)
h [` − 2k]cj,n [`].
`∈Z
Remark 2 The recursive decomposition of f by applying Eq. (14) (known as the Mallat's algorithm [1]),
with ∈ {0, 1}, is based on convolution and downsampling. Remark 3 Expansions from Wj,n to Wj+1,2n ⊕ Wj+1,2n elements summarize in1 : X |`∈Z
cj,n [`]τ2j ` Wj,n = {z
∈Wj,n
}
X |`∈Z
cj+1,2n [`]τ2j+1 ` Wj+1,2n + {z
}
∈Wj+1,2n
X
cj+1,2n+1 [`]τ2j+1 ` Wj+1,2n+1
|`∈Z
{z
∈Wj+1,2n+1
}
Remark 4 Eq. (14) can be used in practice for a decomposition from sample observations (no need to know f(t) for every t ∈ R). In practice, when the input data of the decomposition are the samples of some
function that satis es Shannon's sampling theorem, we implicitly decompose the space U = US of those elements of L2(R) that have their Fourier transforms with support in [−π, π] (by default). The elements of US are said to be band-limited functions. According to Shannon's sampling theorem, US is the space spanned by the translated versions of ΦS(t) = sinc(t) = sin(πt)/πt. 1
Equalities hold in
L2 (R)
sense in these expansions.
A. Atto - Polytech Annecy-Chambry
p. 8
2 1D DISCRETE WAVELET PACKET DECOMPOSITION 2.2 Scaling function, wavelets, wavelet lters
2.2
Scaling function, wavelets, wavelet filters
Remark 5 In Eq. (4), W0 6= Φ in general. When W0 = Φ, we will say that Φ is a scaling function
associated with a wavelet function Ψ = W1. In this case, lters H0 and H1 are called wavelet lters (H0 is associated to Φ and H1 is associated to Ψ). Definition 1 (Wavelet) A wavelet is a regular (wide sense) function Ψ ∈ L1 (R) ∩ L2 (R) satisfying:
Ψ has r vanishing moments, for some r > 1, i.e. Z tm Ψ(t) dt = 0, for every m = 0, 1, . . . , r − 1. R
Ψ is well-localized (with suciently fast decay, say: tΨ(t) ∈ L1 (R) for instance).
W1 and Ψ denote wavelets hereafter
Wavelet functions
A. Atto - Polytech Annecy-Chambry
p. 9
(15)
2 1D DISCRETE WAVELET PACKET DECOMPOSITION 2.2 Scaling function, wavelets, wavelet lters
Some Properties
Scaling lter H0 is low-pass and wavelet lter H1 is high-pass. Wj,n,k is a scaling function if n = 0, a wavelet function if n 6= 0. subspace Wj,n is called a subband, associated with – a Discrete Wavelet Transform (DWT) when n ∈ {0, 1}, – a Discrete Wavelet Packet Transform (DWPT) when n ∈ {0, 1, . . . , 2j } at level j
j is the wavelet decomposition level (scale or resolution parameter) n is the wavelet shift parameter From now on
Filter H1 is a wavelet lter: H1 is associated with a wavelet Ψ. Filter H0 is a scaling lter obtained from QMF condition on (H0 , H1 ) (see Eq. (3)). The decomposition starts from samples of f, where f is assumed to belong to the space US of band-limited
functions: Φ = ΦS = sinc and thus, W0 6= Φ in Eq. (4).
A. Atto - Polytech Annecy-Chambry
p. 10
2 1D DISCRETE WAVELET2.3 PACKET Discrete DECOMPOSITION Wavelet Transform (DWT): signal decomposition examples
2.3
Discrete Wavelet Transform (DWT): signal decomposition examples `Doppler' signal.
`Blocks' signal. 20
`Cusp' signal.
0.5
0.4
`HyChirps' signal.
0.8
20
0.7
15
0.6
10
0.5
5
15 0.3
0.2 10 0.1
5
0
−0.1
0.4
0
0.3
−5
0.2
−10
0.1
−15
0 −0.2
−0.3 −5 −0.4
−10
0
500
1000
1500
2000
2500
−0.5
0
`Blocks' DWT for J = 6. 200
500
1000
1500
2000
2500
0
0
500
`Doppler' DWT for J = 6. 100
1000
1500
2000
−20
2500
`Cusp' DWT for J = 6.
0
500
1000
1500
2000
2500
`HyChirps' DWT for J = 6.
250
100
80
80
150
200 60 60 40
100
150 40
20
50
0
100
20
−20
0
0
50 −40 −20 −60
−50
0 −40
−80
−100
0
500
1000
1500
2000
2500
`Blocks' DWT for J = 10. 250
−100
0
500
1000
1500
2000
2500
−50
0
500
`Doppler' DWT for J = 10. 150
1000
1500
2000
`Cusp' DWT for J = 10. 800
0
500
1000
1500
2000
2500
`HyChirps' DWT for J = 10. 100
700
200
−60
2500
80
100 600 150
60 500 50
100
40
400
50
0
300
20
200
0
0
−50 100 −50
−20 0 −100
−100
−150
−40
−100
0
500
1000
1500
2000
2500
−150
0
500
1000
1500
2000
2500
−200
0
500
1000
1500
2000
2500
−60
0
500
1000
1500
2000
2500
Figure 2: Examples of signals and their DWT representations. The DWT concerns the resolution levels j = 1, 2, . . . , J where J is either 6 or 10.
A. Atto - Polytech Annecy-Chambry
p. 11
2 1D DISCRETE WAVELET PACKET 2.4 DWT DECOMPOSITION and frequencies: limit case of the Shannon wavelet packets
2.4
DWT and frequencies: limit case of the Shannon wavelet packets
The decomposition space U = US (π-band-limited Paley-Wiener space ) by using Shannon scaling and wavelet lters is given by Figure 3. The Shannon wavelet packet tree induces a frequency re-ordering. US
[−π, π]
1 =0 S W1,0 [0, π2 ] S W2,0 [0, π4 ] S W3,0 [0, π8 ]
S W3,1 π π [8, 4]
S W1,1 π [ 2 , π]
2 =1 S W2,1 π π [4, 2]
S W3,2 3π π [ 8 , 2]
3 =1
S W3,3 π 3π [4, 8 ]
S W2,2 3π [ 4 , π] S W3,4 7π [ 8 , π]
S W2,3 π 3π [2, 4 ]
S W3,5 3π 7π [4, 8]
S W3,6 π 5π [2, 8 ]
S W3,7 5π 3π [8, 4]
S S Figure 3: Shannon wavelet packet decomposition tree. The positive part of the support of IF Wj,n is indicated under each node Wj,n .
When the Shannon wavelet lters are used to decompose the Paley Wiener space, then IF ΦS = 1l[−π,π] and S IF Wj,n (ω) = 2j 1l∆j,G(n) (ω).
(16)
+ − + where 1lK denotes the indicator function of the set K and ∆j,G(n) = ∆− j,G(n) ∪ ∆j,G(n) , with ∆j,G(n) = −∆j,G(n) ,
∆+ j,G(n)
G(n)π (G(n) + 1)π = , . 2j 2j
(17)
The function G in Eq. (17) is a permutation de ned by G(0) = 0 and G(2` + ) = 3G(`) + − 2
G(`) + 2
where bzc is the largest integer less than or equal to z and ∈ {0, 1}. A. Atto - Polytech Annecy-Chambry
p. 12
,
(18)
3 2-D SEPARABLE WAVELETS AND WAVELET PACKETS
3
2-D separable wavelets and wavelet packets
3.1
Separability
If we consider a 2-D separable extension from 1D wavelet packet functions, then Wj,[n1 ,n2 ] = Wj,n1 ⊗ Wj,n2
where Wj,n1 and Wj,n2 are 1D wavelet packet subspaces. The wavelet packet subband Wj,[n1 ,n2 ] is, by construction, the closure of the L2 (R2 ) subspace spanned by the orthornormal system of wavelet packet functions :
τ[2j k1 ,2j k2 ] Wj,[n1 ,n2 ] : (k1 , k2 ) ∈ Z2 .
Due to separability, the wavelet packet function Wj,[n1 ,n2 ] satis es: IF Wj,[n1 ,n2 ] (ω1 , ω2 ) = IF Wj,n1 (ω1 ) IF Wj,n2 (ω2 ).
(19)
Splitting a given (input) functional space U = W0,[0,0] ⊂ L2 (R2 ) into orthogonal subspaces (subbands) is then performed by using: W0,[0,0] = W1,[0,0] ⊕ W1,[0,1] ⊕ W1,[1,0] ⊕ W1,[1,1] ,
and, for j > 1, Wj,[n1 ,n2 ] = Wj+1,[2n1 ,2n2 ] ⊕ Wj+1,[2n1 ,2n2 +1] ⊕ Wj+1,[2n1 +1,2n2 ] ⊕ Wj+1,[2n1 +1,2n2 +1]
A. Atto - Polytech Annecy-Chambry
p. 13
3 2-D SEPARABLE WAVELETS AND WAVELET PACKETS 3.2 2D DWT and DWPT input space and trees
3.2
2D DWT and DWPT input space and trees
An example of wavelet packet representation is given by the tree of Figure 4. The space U is the L2 (R) subspace spanned by an orthonormal system
τ[k1 ,k2 ] Φ : (k1 , k2 ) ∈ Z2
where τ is the 2D shift operator : τ[k1 ,k2 ] f(t1 , t2 ) = f(t1 − k1 , t2 − k2 ).
When the input space U is the Paley Wiener space of functions having their Fourier transforms with supports in [−π, π] × [−π, π], then Φ is the 2D sinc function: Φ(t1 , t2 ) = sinc(t1 , t2 ) =
sin(πt1 ) sin(πt2 ) π2 t1 t2
∈ L2 (R2 )
U = W0,[0,0]
W1,[0,0]
W2,[0,0]
W3,[0,0]
W3,[0,1]
W3,[1,0]
W2,[0,1]
W3,[1,1]
W3,[2,0]
p. 14
W1,[1,0]
W1,[1,1]
W2,[1,0]
Figure 4: Example of 2D Wavelet packet tree.
A. Atto - Polytech Annecy-Chambry
W1,[0,1]
W3,[2,1]
W3,[3,0]
W2,[1,1]
W3,[3,1]
3 2-D SEPARABLE WAVELETS AND WAVELET PACKETS 3.3 Shannon DWT and DWPT trees
3.3
Shannon DWT and DWPT trees
When the Shannon ideal paraunitary lters are used to decompose the Paley Wiener space, the Fourier transform S of the Shannon wavelet packet function Wj,n satis es:
S IF Wj,n (ω1 , ω2 ) = 2j 1l∆j,G(n1 ) ×∆j,G(n2 ) (ω1 , ω2 ).
(20)
WS 0,[0,0]
[0,π]×[0,π]
WS
i h i h 1,[0,0] × 0,π 0,π 2 2
WS
h 2,[0,0] i h i 0,π × 0,π 4 4
WS
i h i h 3,[0,0] 0,π × 0,π 8 8
WS
i h i h 3,[0,1] 0,π ×π ,π 8 8 4
WS
h 2,[0,1] i h i 0,π ×π ,π 4 4 2
WS 3,[1,0] i h i π ,π × 0,π 8 4 8
h
WS
i h i h 1,[0,1] ×π ,π 0,π 2 2
WS 2,[1,0] i h i π ,π × 0,π 4 2 4
h
WS 1,[1,0] i h i π ,π × 0,π 2 2
h
WS 1,[1,1] i h i π ,π × π ,π 2 2
h
WS 2,[1,1] i h i π ,π × π ,π 4 2 4 2
h
WS 3,[1,1] i h i π ,π × π ,π 8 4 8 4
h
Figure 5: Shannon 2D DWT tree and positive part of the support for the Fourier transforms of the Shannon wavelet functions.
A. Atto - Polytech Annecy-Chambry
p. 15
REFERENCES
REFERENCES
WS 0,[0,0]
[0,π]×[0,π] WS 0,0
WS
WS
WS
h 1,[0,0] i h i 0,π × 0,π 2 2
h 1,[0,1] i h i 0,π ×π ,π 2 2
WS 1,0
WS 1,1
WS
h 2,[0,0] i h i 0,π × 0,π 4 4
i h i h 2,[0,1] ×π ,π 0,π 4 4 2
WS 2,0
WS 2,1
WS 2,[1,0] i h i π ,π × 0,π 4 2 4 S W 2,2
h
WS 2,[1,1] i h i π ,π × π ,π 4 2 4 2 S W 2,3
h
WS
WS
h 2,[0,2] i h i 0,π × 0,π 4 4
i h i h 2,[0,3] ×π ,π 0,π 4 4 2
WS 2,4
WS 2,5
WS 2,[1,2] i h i π ,π × 0,π 4 2 4 S W 2,6
h
WS 1,[1,0] i h i π ,π × 0,π 2 2 WS 1,2
WS 1,[1,1] i h i π ,π × π ,π 2 2 WS 1,3
h
WS 2,[1,3] i h i π ,π × π ,π 4 2 4 2 S W 2,7
h
WS
WS
h 2,[2,0] i h i 0,π × 0,π 4 4
i h i h 2,[2,1] ×π ,π 0,π 4 4 2
WS 2,8
WS 2,9
WS 2,[3,0] i h i π ,π × 0,π 4 2 4 S W 2,10
h
h
WS 2,[3,1] i h i π ,π × π ,π 4 2 4 2 S W 2,11
h
WS
WS
h 2,[2,2] i h i 0,π × 0,π 4 4
i h i h 2,[2,3] ×π ,π 0,π 4 4 2
WS 2,12
WS 2,13
WS 2,[3,2] i h i π ,π × 0,π 4 2 4 S W 2,14
h
Figure 6: Shannon 2D DWPT tree and positive part of the support for the Fourier transforms of the Shannon wavelet functions.
References [1] S. Mallat, A wavelet tour of signal processing, second edition. Academic Press, 1999. [2] I. Daubechies, Ten lectures on wavelets. SIAM, Philadelphie, PA, 1992.
See image decomposition example at next page ;
A. Atto - Polytech Annecy-Chambry
p. 16
WS 2,[3,3] i h i π ,π × π ,π 4 2 4 2 S W 2,15
h
REFERENCES
3.4
3.4 2D Discrete Wavelet Transform: image decomposition example
2D Discrete Wavelet Transform: image decomposition example
A. Atto - Polytech Annecy-Chambry
p. 17
