>>> Signal and Image processing in Remote Sensing >>> Introduction Name: Date:

[~]$ _

Mathieu Fauvel (UMR 1201 DYNAFOR INRA - INPT/ENSAT) 2016

[1/89]

[~]$ _

[2/89]

>>> Outline 1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[~]$ _

[3/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[1. Introduction]$ _

[4/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[1. Introduction]$ _

[5/89]

>>> Definition

Remote sensing1 : Remote sensing provides information about landscapes. This information is carried out by the electromagnetic energy and is usually formated in terms of image data. Remote sensing allows to: ? Update and supervise landscapes in known areas, ? Get prior information about landscapes of an unknown areas

1

M. Robin, Télédétection.

[1. Introduction]$ _

Des satellites aux SIG, NATHAN Université. [6/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[1. Introduction]$ _

[7/89]

>>> Acquisition process

Keshava, N.; Mustard, J.F., "Spectral unmixing," Signal Processing Magazine, IEEE, vol.19, no.1, pp.44,57, Jan 2002 [1. Introduction]$ _

[8/89]

>>> Acquisition techniques

Passive Reflected (Solar) and emitted radiation (UV → near IR)

Active Reflection of an emitted signal

Emitted radiation (Thermal IR)

[1. Introduction]$ _

[9/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[1. Introduction]$ _

[10/89]

>>> Maps !

[1. Introduction]$ _

[11/89]

>>> Agriculture ? ? ? ? ?

Classification of agricultural fields, Health monitoring, Mapping of agricultural practices, Verification of funding, ...

Figure: Fields classification from a multispectral SPOT-4 image. [1. Introduction]$ _

[12/89]

>>> Forestry ? ? ? ? ?

Discrimination of the canopy, Biomass estimation, Species inventory, Fire management, ...

[1. Introduction]$ _

[13/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology ? Mapping of water resources, wetlands ? Monitoring of flooding, tsunami ... ? ...

Ucayali River [1. Introduction]$ _

[14/89]

>>> Hydrology

[1. Introduction]$ _

[15/89]

>>> Geography ? ? ? ?

Town planning, Extraction of road network, Greens spaces ...

Thematic map of the center of Reykjavik, Iceland.

[1. Introduction]$ _

IKONOS.

[16/89]

>>> Geography

[1. Introduction]$ _

[17/89]

>>> Geography

[1. Introduction]$ _

[17/89]

>>> Meteorology ? Monitoring of atmospheric masses, ? Weather forecasting, ? Natural disasters (storm, cyclone) ? ...

Channel C: visible.

[1. Introduction]$ _

Channel D: thermal.

Channel E: mid infrared

[18/89]

>>> Meteorology

1979

2013

http://earthobservatory.nasa.gov/Features/WorldOfChange/ozone.php [1. Introduction]$ _

[19/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[2. Physics of remote sensing]$ _

[20/89]

>>> Electromagnetic wave ~ B

λ

x

Direction

E~ ? An electromagnetic wave is a pertubation of the electric and magnetic field which propagates through space. ~ ~ ? E: Electric field. B: Magnetic field. ? λ: wavelength. Minimal distance between 2 point of the space for which E~ and ~ recover the same values. B ? c: speed propagation of light ( 3.108 ) c ? Frequency ν = λ Blue

[2. Physics of remote sensing]$ _

Near infrared

λ

[21/89]

>>> Electromagnetic spectrum

? Definition : it represents the range of all possible wavelength ? Electromagnetic radiance: ? ? ? ?

Reflected radiance (sun): visible and near infra red (passive remote sensing). Reflected and emitted radiance : mid infrared Emitted radiance: thermal infra red. Artificial radiance (radar): high frequency (active remote sensing).

[2. Physics of remote sensing]$ _

[22/89]

>>> Reflectance or reflectivity

Reflectance: it is the capacity of a surface to reflect the incident electromagnetic energy ρ(λ, sensor pos., sun pos.) =

[2. Physics of remote sensing]$ _

Reflected energy Received energy

[23/89]

>>> Spectral reflectance curves Spectral reflectance curves or spectral signature:

"Radiometric identity card".

Oak 1

reflectance

0.8

0.6

0.4

0.2

0 0.4

0.6

[2. Physics of remote sensing]$ _

0.8

1

1.2

1.4 λ (μm)

1.6

1.8

2

2.2

2.4 [24/89]

>>> Atmospheric effect 1/2 ? Absorption:

some wavelength are absorbed

? Scattering:

molecules change wavelength direction (blue sky)

? ? ? ?

UV et λ < 0.3μm are absorbed by O3 . Visible and near infra red are almost transmitted by the atmosphere mid and thermal infrared: complete absorption for some λ. Perfect transmission for microwave (cloud ...)

Figure: Atmosphere transmission. [2. Physics of remote sensing]$ _

[25/89]

>>> Atmospheric effect 2/2

1 0.8 0.6 0.4 0.2 0

[2. Physics of remote sensing]$ _

0.5

1

1.5

2

2.5

[26/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[3. Spectral signature]$ _

[27/89]

>>> Spectral signatures

? The reflectance characterizes each material ? ? ? ?

Mineral, rocks and soils, Vegetation, Water, Man made surfaces.

? Field measure:

measure of the reflectance in particular condition

? Field spectral signature can be used to calibrate sensor. ? However, difference between field measurement and remote sensing acquisition exist.

[3. Spectral signature]$ _

[28/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[3. Spectral signature]$ _

[29/89]

>>> Spectral signature of vegetation Typical signature: Healthy vegetation (high photosynthesis) ? Absorption in blue and red, ? Visible to near infrared: increase of the reflectance, ? Mid infrared: depends on the free water in the leafs.

[3. Spectral signature]$ _

[30/89]

>>> Spectral signature of vegetation

Factors modifying the reflectance: ? Leaf thickness ? Leaf age ? Water content ? Health condition ? ...

Figure: Spectral signature of “Coast redwood”: green and dry.

[3. Spectral signature]$ _

[31/89]

>>> Spectral signature of vegetation

April

[3. Spectral signature]$ _

October

[32/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[3. Spectral signature]$ _

[33/89]

>>> Spectral signature of water

Water appears: ? Blue - scattering, ? Green - water with chlorophyll, ? Yellow-Brown - turbid water. Under some conditions, specular reflection may happen. [3. Spectral signature]$ _

[34/89]

>>> Spectral signature of water

Yellow river, China.

Sea of Azov, Russia. [3. Spectral signature]$ _

[35/89]

>>> Man made surfaces Very specific for different materials

[3. Spectral signature]$ _

[36/89]

>>> Man made surfaces Very specific for different materials

[3. Spectral signature]$ _

[36/89]

>>> Man made surfaces Very specific for different materials

[3. Spectral signature]$ _

[36/89]

>>> Man made surfaces Very specific for different materials

[3. Spectral signature]$ _

[36/89]

Where is the: fake? e>>> anomalie petit personnage en LEGO

[3. Spectral signature]$ _

[37/89]

>>> Where is the fake?

473 nm

Images en niveau de gris à quelques longueurs d'onde [3. Spectral signature]$ _

[37/89]

>>> Where is the fake?

547 nm

Images en niveau de gris à quelques longueurs d'onde [3. Spectral signature]$ _

[37/89]

>>> Where is the fake?

621 nm

Images en niveau de gris à quelques longueurs d'onde [3. Spectral signature]$ _

[37/89]

>>> Where is the fake?

681 nm

Images en niveau de gris à quelques longueurs d'onde [3. Spectral signature]$ _

[37/89]

>>> Where is the fake?

473 nm 547 621 681 770

Images en niveau de gris à quelques longueurs d'onde [3. Spectral signature]$ _

[37/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[4. Remote sensing imagery]$ _

[38/89]

>>> Nature of remote sensing images A remote sensing image is a sampling of a spatial, spectral and temporel process

1 0.8 0.6 0.4 0.2 0

[4. Remote sensing imagery]$ _

500 600 700 800 900

[39/89]

>>> Nature of remote sensing images A remote sensing image is a sampling of a spatial, spectral and temporel process

1 0.8 0.6 0.4 0.2 0

[4. Remote sensing imagery]$ _

500 600 700 800 900

[39/89]

>>> Nature of remote sensing images A remote sensing image is a sampling of a spatial, spectral and temporel process

1 0.8 0.6 0.4 0.2 0

[4. Remote sensing imagery]$ _

500 600 700 800 900

[39/89]

>>> Nature of remote sensing images A remote sensing image is a sampling of a spatial, spectral and temporel process

1 0.8 0.6 0.4 0.2 0

500 600 700 800 900

t y y h ar ar rc ru nu b a Ma e J F [4. Remote sensing imagery]$ _

l ri Ap

y Ma

ne Ju

y ll Ju

t

s gu

Au

p Se

te

er mb

r be

to Oc

ve No

er mb

er mb

ce De

[39/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[4. Remote sensing imagery]$ _

[40/89]

>>> Grayscale digital images An image is made of pixels (picture element), which is the smallest homogeneous surface of the digital image. !"#

$%%

$!#

!%#

!&#

!&#

!%%

!'#

!!#

()*+,-./!%0 !"""""""""""""""""""""#$%&''&"(&")*+,"(&"-./,"""""""""""""""""011

Each pixel has a value (scalar, vector- or matrice-value), a spatial position and a size. [4. Remote sensing imagery]$ _

[41/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[4. Remote sensing imagery]$ _

[42/89]

>>> Spatial resolution Ability to distinguish two closed objects !"#$%&!

,-./01.23%4!

'%(%)*%+

'%(%5%+

SPOT XS (20m) [4. Remote sensing imagery]$ _

Quickbird MS (4m)

607879%":8

Ikonos Pan (1m) [43/89]

>>> Spatial resolution and coverage

[4. Remote sensing imagery]$ _

[44/89]

>>> Spatial coverage

$%"&

!"##

/01+2

'()*+,)-.

01+2

!"#$%&'()("*(+,--.(/0012

[4. Remote sensing imagery]$ _

[45/89]

>>> Spectral resolution ? It is the smallest difference in wavelength that the sensor can record. ? Spectral and spatial resolution are linked ( ΔΔs = k) λ

? Panchromatic image (one band): ? Multispectral image (few bands):

0.4

0.4

? Hyperspectral image (hundred of bands):

[4. Remote sensing imagery]$ _

0.7

0.5

0.4

0.6

0.7

0.7

[46/89]

>>> Spectral resolution 1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0 400

600

800

1000

0 400

600

(a) 1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

600

800

(c) Figure: Spectral sampling : hyper VS multi. [4. Remote sensing imagery]$ _

1000

800

1000

(b)

1

0 400

800

1000

0 400

600

(d)

(a) hyperspectral, (b) multispectral et (c) panchromatic.

(d) [47/89]

>>> Spectral Resolution

Pleiades sensor [4. Remote sensing imagery]$ _

[48/89]

>>> Radiometric resolution ? It is the smallest difference in intensity that the sensor can record. ? It is related to the number of bits used to store the data ? Base 2 encoding on 1 octet (8bits) of 79: 7

5

01001111

79 = 0 × 2 + 1 × 2 + 0 × 2 + 0 × 2 + 1 × 23 + 1 × 22 + 1 × 21 + 1 × 20 ? 8 bits : ? 16 bits:

6

4

256 values (0→255) 65536 values

3 2 1

0

[4. Remote sensing imagery]$ _

1 1.5 2

3

4

5

[49/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Temporal resolution

[4. Remote sensing imagery]$ _

[50/89]

>>> Displaying color images ? Human eye sensitivity to wavelength

? Definition of colors:

[4. Remote sensing imagery]$ _

[51/89]

>>> Displaying color images Examples of additive color composition, for an 8 bits image: Blue Green Red Color

[4. Remote sensing imagery]$ _

0

0

0

255

255

255

255

0

0

0

255

0

0

0

255

0

255

255

255

255

0

255

0

255

127

0

0

25

0

0

0

127

127

127

127

127

25

25

25

200

200

200 [52/89]

>>> Displaying color images

? Need to choose the spectral bands, at most three. ? Different spectral compositions lead to different color images... ? One band ⇒ grayscale image ? 3 bands ⇒ color image [4. Remote sensing imagery]$ _

[53/89]

>>> Composite colors Two conventional colorisations: True color λblue

[4. Remote sensing imagery]$ _

λgreen

False color λred

λgreen

λred

λinfra

red

[54/89]

>>> Composite colors

0.45-0.52 μm

[4. Remote sensing imagery]$ _

0.52-0.60 μm

0.63-0.69 μm

0.76-0.90μm

[55/89]

>>> Composite colors 0.45-0.52 μm

[4. Remote sensing imagery]$ _

0.52-0.60 μm

0.63-0.69 μm

0.76-0.90μm

[55/89]

>>> Composite colors 0.45-0.52 μm

[4. Remote sensing imagery]$ _

0.52-0.60 μm

0.63-0.69 μm

0.76-0.90μm

[55/89]

>>> Composite colors 0.45-0.52 μm

[4. Remote sensing imagery]$ _

0.52-0.60 μm

0.63-0.69 μm

0.76-0.90μm

[55/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[5. Images Processing]$ _

[56/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[5. Images Processing]$ _

[57/89]

>>> Geometric correction Systematic distortion: ? Hearth rotation ? Viewing angle ? ...

[5. Images Processing]$ _

[58/89]

>>> Geometric correction

[5. Images Processing]$ _

[59/89]

>>> Geometric correction

Non-systematic distortions: ? Changes in satellite speed, altitude ? Relief

[5. Images Processing]$ _

[60/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[5. Images Processing]$ _

[61/89]

>>> Normalized Difference Vegetation Index (NDVI)

? Principle: “The chlorophyll has a high absorption in the red part of the electromagnetic spectrum and a high reflectance in the near-infrared part.” NDVI =

(IR − R) (IR + R)

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

B NDVI =

V (0.95−0.1) (0.95+0.1)

[5. Images Processing]$ _

R = 0.81

IR

B NDVI =

V (0.625−0.6) (0.625+0.6)

R = 0.2

IR

B NDVI =

V (0.1−0.2) (0.1+0.2)

R

IR

= −0.33

[62/89]

>>> Normalized Difference Vegetation Index (NDVI)

[5. Images Processing]$ _

[63/89]

>>> Histogram

An histogram represents the number of pixels for each intensity level

4

x 10 12 10 8 6 4 2 0 −1

[5. Images Processing]$ _

−0.5

0

0.5

1

[64/89]

>>> Contrast enhancement The contrast is defined as the difference between the maximum and the minimum intensity value of an image.

4

3

4

x 10

x 10

2.5

2

2

1.5

1.5 1 1 0.5

0.5 0 0 [5. Images Processing]$ _

50

100

150

200

250

0 0

50

100

150

200

250 [65/89]

>>> Contrast enhancement The contrast is defined as the difference between the maximum and the minimum intensity value of an image.

4

3

4

x 10

x 10

2.5

2

15000

2 10000

1.5

1.5 1 1

5000

0.5

0.5 0 0 [5. Images Processing]$ _

50

100

150

200

250

0 0

50

100

150

200

250

0 0

50

100

150

200

250 [65/89]

>>> Contrast enhancement The contrast is defined as the difference between the maximum and the minimum intensity value of an image.

4

3

4

x 10

x 10

2.5

2

15000

2 10000

1.5

1.5 1 1

5000

0.5

0.5 0 0 [5. Images Processing]$ _

50

100

150

200

250

0 0

50

100

150

200

250

0 0

50

100

150

200

250 [65/89]

>>> Classification ? Objectives: label each pixel to a particular thematic class (water, meadow, ...) ? Supervised classification: Thematic classes are identified by an operator; spectral properties (mean, variance ...) are learned by some algorithms (Gaussian mixture model, neural nets, SVM ...); the whole image is classified. ? Unsupervised classification/Clustering: Pixels are clustered without any a priori (however, the number of cluster is usually provided by an operator)

Auxiliary Decision

data Training of algorithm

rules Perform the actual classification

Thematic map

Remote sensing data

[5. Images Processing]$ _

[66/89]

>>> Supervised Classification Vector representation 1 0.8



0.6

−→

0.4 0.2 0

 0.225   0.169 x=  ∈ R4 0.698 0.567

500 600 700 800 900

Statistical modelization ? Pixels x1 , . . . , xn to be classified are realization of a random vector X valued in Rd . ? Values y1 , . . . , yn are realization of a random variable Y valued in {1, . . . , C}.

[5. Images Processing]$ _

[67/89]

>>> Supervised Classification

K-nearest neighbors (K-nn) ? C classes:

ω1 , . . . , ωC , with n samples xj per class

? A pixel x is classified to ω if its (k-)nearest neighbors belongs to ω . ? The decision rule is (Euclidean distance): g (x) = min kx − xj k2 j

[5. Images Processing]$ _

[68/89]

>>> Supervised Classification

Distance to the mean ? C classes:

ω1 , . . . , ωC

? A pixel x is classified to ω if d(μ , x) < d(μj , x), ∀j 6= . ? The estimator is μ =

n 1 X

n

x

=1

? The decision rule is (Euclidean distance) g (x) = (x − μ )t (x − μ )

[5. Images Processing]$ _

[69/89]

>>> Supervised Classification Maximum a-posteriori: ? C classes: ω1 , . . . , ωC ? A pixel x is classified to ω si p(ω |x) > p(ωj |x), ∀j 6= . ? Bayes:

p(ω |x) = p(x|ω )p(ω )/ p(x)

? The decision rule becomes: p(x|ω )p(ω ) > p(x|ωj )p(ωj ), ∀j 6= . ? The logarithm is used g (x) = ln(p(x|ω )) + ln(p(ω )) ? Under Gaussian assumption for the conditional probability: p(x|ω ) = (2π)−d/ 2 | |−1/ 2 exp(−0.5(x − m )t −1  (x − m )) ? Finally, the decision rule is: g (x) = ln(p(ω )) −

[5. Images Processing]$ _

1 2

ln(| |) −

1 2

(x − m )t −1  (x − m )

[70/89]

>>> Supervised classification

b2

b2

b2

b1

b1

K-nn

[5. Images Processing]$ _

Mean distance

b1

MAP

[71/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V F E

[72/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V 1 F E

[72/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V 1 F 1 E

[72/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V 2 F 1 E

[72/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V 2 1 F 1 E

[72/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V 2 1 F 1 E 1

[72/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V 2 1 F 1 E 2

[72/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V 2 1 F 1 E 1 2

[72/89]

>>> Confusion matrix Confusion Matrix: data.

Compute the agreement between the classifier and the reference V

V

F

F

V

V

V

F

E

V

V

F

E

E

E

V

V

E

E

V

Classification

F F E

Reference data

Classification

Construction :

[5. Images Processing]$ _

Reference data V F E V 2 1 0 F 0 1 0 E 1 0 2

[72/89]

Classification

>>> Confusion matrix

1 . . . C

Reference data 1 ... C n11 ... n1C . .. . . . nC1 nCC pr1 ... prC

p1 . . . pC

Measure of agreement PC

? Overall accuracy :

=1

n

n ? User accuracy : Percentage of pixels classified to class  that belong actually to class  in the reference data. n p = PC n j=1 j ? Producer accuracy : Percentage of pixels of the reference data belonging to class  that are correctly classified to class . n pr = PC n j=1 j [5. Images Processing]$ _ [73/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[5. Images Processing]$ _

[74/89]

>>> Spatial neighborhood The neighborhood of a given pixel is the set of pixels that are connected to it. For a flat (grayscale) image : x−1,−1 x0,−1 x1,−1

x−1,−1 x0,−1 x1,−1

x−1,0 x0,0

x1,0

x−1,0 x0,0

x1,0

x−1,1 x0,1

x1,1

x−1,1 x0,1

x1,1

4-connected

8-connected

Wide range of processing are based on pixel neighborhood ? De noising, ? Texture analysis, ? Edges detection, ? Pattern recognition, ? ... [5. Images Processing]$ _

[75/89]

>>> Template filters Steps: 1. Define the template G: 4/8-connected and size 2. Define the processing ƒ on the neighborhood. If ƒ is linear ↔ convolution. 3. Scan all the pixels: ƒ

xj = ƒ (x1 , . . . , xN ), xn ∈ G(, j) 27

26

25

24

24

23

25

24

23

23

22

21

23

23

22

21

20

20

22

21

20

20

19

18

21

20

19

18

18

17

20

19

18

17

17

16

23

Filter Max [5. Images Processing]$ _

[76/89]

>>> Example Max & Min

Min

[5. Images Processing]$ _

Original

Max

[77/89]

>>> Mean filter 

1  ? G = 1 1

1 1 1

?  ƒ (, y) =

1 9

 1  1, for a 3 × 3 neighborhood. 1 P1

ƒ

,j=−1

( + , y + j) ƒ

? (,y) = (,y) ∗ G(,y) ⇔ (,) = (,) × G(,) ? Mean filter = low pass filter

G(, y)

[5. Images Processing]$ _

|G(, )|

[78/89]

>>> Example

[5. Images Processing]$ _

[79/89]

>>> Edge detection 1/4 ? What an edge is?

300 250 200 150 100 50 0 0 ? First approximation:

[5. Images Processing]$ _

50

100

150

200

a strong variation of the pixel intensity

[80/89]

>>> Edge detection 2/4 The variation is assessed with the first derivative of the image ƒ 0 () = lim

ƒ ( + h) − ƒ ()

h→0 h Template approximation of the first derivative (less robust to most robust)

0

0

0

-1

0

1

-1

0

1

0

-1

1

-1

0

1

-2

0

2

0

0

0

-1

0

1

-1

0

1

Gradient

Prewitt

Sobel abs(ƒ 0 ())

ƒ ()

 abs(ƒ 0 ())

ƒ ()

[5. Images Processing]$ _







[81/89]

>>> Edge detection 3/4

The local gradient is equivalent to an high pass filter:

G(, y)

[5. Images Processing]$ _

|G(, )|

[82/89]

>>> Edge detection 4/4 Usually, the gradient is computed by applying the template in each direction: -1

0

1

-1

-2

-1

-2

-1

0

0

-1

-2

-2

0

2

0

0

0

-1

0

1

1

0

-1

-1

0

1

1

2

1

0

1

2

2

1

0

Vertical

[5. Images Processing]$ _

Horizontal

NE-SW

NW-SE

[83/89]

>>> Edge detection 4/4 Usually, the gradient is computed by applying the template in each direction: -1

0

1

-1

-2

-1

-2

-1

0

0

-1

-2

-2

0

2

0

0

0

-1

0

1

1

0

-1

-1

0

1

1

2

1

0

1

2

2

1

0

Vertical

[5. Images Processing]$ _

Horizontal

NE-SW

NW-SE

[83/89]

>>> Edge detection 4/4 Usually, the gradient is computed by applying the template in each direction: -1

0

1

-1

-2

-1

-2

-1

0

0

-1

-2

-2

0

2

0

0

0

-1

0

1

1

0

-1

-1

0

1

1

2

1

0

1

2

2

1

0

Vertical

[5. Images Processing]$ _

Horizontal

NE-SW

NW-SE

[83/89]

>>> Edge detection 4/4 Usually, the gradient is computed by applying the template in each direction: -1

0

1

-1

-2

-1

-2

-1

0

0

-1

-2

-2

0

2

0

0

0

-1

0

1

1

0

-1

-1

0

1

1

2

1

0

1

2

2

1

0

Vertical

[5. Images Processing]$ _

Horizontal

NE-SW

NW-SE

[83/89]

>>> Edge detection 4/4 Usually, the gradient is computed by applying the template in each direction: -1

0

1

-1

-2

-1

-2

-1

0

0

-1

-2

-2

0

2

0

0

0

-1

0

1

1

0

-1

-1

0

1

1

2

1

0

1

2

2

1

0

Vertical

[5. Images Processing]$ _

Horizontal

NE-SW

NW-SE

[83/89]

>>> Edge detection 4/4 Usually, the gradient is computed by applying the template in each direction: -1

0

1

-1

-2

-1

-2

-1

0

0

-1

-2

-2

0

2

0

0

0

-1

0

1

1

0

-1

-1

0

1

1

2

1

0

1

2

2

1

0

Vertical

[5. Images Processing]$ _

Horizontal

NE-SW

NW-SE

[83/89]

>>> Texture ? Definition: “spatial structure made of spatial partterns, each of them having a random aspect”. ? Texture is used to describe surfaces (smooth, rough, regular ...)

? Basically, textures are characterized by local measures (template filtering) ? Variance, ? Dynamic, ? Entropy.

[5. Images Processing]$ _

[84/89]

1. Introduction Definition Satellite remote sensing Thematic applications 2. Physics of remote sensing 3. Spectral signature Vegetation Water 4. Remote sensing imagery Digital image Characteristics of remote sensing images 5. Images Processing Pre-processing Processing in the spectral domain Processing in the spatial domain Image registration

[5. Images Processing]$ _

[85/89]

>>> Image registration ? Definition: “Image registration is the process of overlaying two or more images of the same scene taken at different times, from different viewpoints, and/or by different sensors”. ? Very important process in remote sensing (multitemporal analysis ...) ? The problem is to find a spatial transformation mapping one image to another one.

3’

2’ 4’4

1

[5. Images Processing]$ _

3

1’

2

[86/89]

>>> Image registration: Minimization of the RMSE ¦ P
2 © n ? ƒ such as min 1n =1 0 − ƒ ( ) ? Polynomial transform:

ƒ ( ) = α0 + α1  + α2h + α3  h   α0  ” — α1   ƒ ( ) = 1  h h    α 2  α3

? RMS



1 n

0

x − Xα

‹2

avec x

0

  01 1  0  1  2   et X =  = .   .  . 0 n 1

 1  2 . . .  n

h1 h2 hn

 h1  1  h2  2   

hn  n

? By deriving with respect to à α and at the optimal ‹ € Š−1 2  0 0 Xt x − Xt x − Xα = 0 ⇒ α = Xt X n

[5. Images Processing]$ _

[87/89]

>>> Image registration

Linear image registration

[5. Images Processing]$ _

Non-linear image registration

[88/89]

>>> Registration:

[5. Images Processing]$ _

map - image

[89/89]