A database of polarimetric and multispectral images in the visible and NIR regions P IERRE -J EAN L APRAY, L UC G ENDRE , A LBAN F OULONNEAU , AND L AURENT B IGUÉ IRIMAS, U NIVERSITÉ DE H AUTE -A LSACE , M ULHOUSE , F RANCE

I NTRODUCTION

I MAGING P IPELINE

Multi-band polarization imaging, by mean of analyzing spectral and polarimetric data simultaneously, is a good way to improve the the information recovered from a scene. This work presents a database of polarimetric and multispectral images that combine visible and near-infrared (NIR) information. An experimental setup is built around a dual-sensor camera.

ˆ −1 M

12 Raw Image Acquisitions

Linearization + FPN & PNRU corrections

Image Registration

Multispectral generation from dual-RGB reassembly

ˆ R

Spectral Calibration

Spectral Estimation

Polarimetric Calibration

Stokes Estimation

−1 ˆ Ai

D ATABASE

Sˆi

R ESULTS

C ONTRIBUTIONS 1. A practical imaging pipeline, including a calibration procedure, for recovering spatial, spectral and polarimetric data from a set of photographs, 2. A database of multispectral and polarimetric images of various materials in the visible and near- infrared part of the spectrum.

(a) Macbeth classic

(d) Painting

(b) Macbeth enhancement

(a) I0-blue filter

(b) I45-blue filter

(c) I90-blue filter

(d) I135-blue filter

(e) I0-yellow filter

(f) I45-yellow filter

(g) I90-yellow filter

(h) I135-yellow filter

(i) I0-NIR

(j) I45-NIR

(k) I90-NIR

(l) I135-NIR

(c) Glass

(e) Fake & real leaves

(f) Potery

Figure 5: 12 raw images for the "Glass" scene after registration.

E XPERIMENTAL S ETUP (g) Food

(h) Fabrics

(a) s0 band 1

(b) s0 band 2

(c) s0 band 3

(d) s0 band 4

(e) s0 band 5

(f) s0 band 6

(g) s1 band 1

(h) s1 band 2

(i) s1 band 3

(j) s1 band 4

(k) s1 band 5

(l) s1 band 6

(m) s2 band 1

(n) s2 band 2

(o) s2 band 3

(p) s2 band 4

(q) s2 band 5

(r) s2 band 6

(i) Liquid

(j) Knife

Figure 6: 6-band linear Stokes images s0 , s1 and s2 .

Figure 3: sRGB visualization of the scenes. sRGB are reconstructed from a linear transform of the 6-band multispectral images after polarization component removing.

(a) AOLP band 3

(a) Painting

(b) Fake & real leaves

(c) Potery

Figure 1: Experimental setup and optical path.

(b) DOLP band 3

(c) DOLP zoom

Figure 7: Glass scene: (a)-(b) Visualization of Angle and Degree Of Linear Polarization for band 3. (c) Potential defect detection application.

Figure 4: Simple RGB visualizations without any spectral and polarization processing.

F UTURE D IRECTIONS (a)

(c)

(b)

(d)

Figure 2: (a) RGB/NIR camera sensitivities. (b) Transmittance of the bandpass filters. (c) RGB reassembly using blue-green (bg) and yellow (y) filters. (d) Final camera responses.

• The spectral signature is recovered using an RGB/NIR camera plus a couple of bandpass filters. Multispectral images are constructed from the dual-RGB method [1]. • The polarimetric feature is achieved using rotating linear polarization filters in front of the camera at four different angles (0, 45, 90 and 135 degrees).

(a) AOLP band 6

(b) AOLP zoom and brushstrokes direction

(c) Reflectance estimation for 3 pigments

1. Improve pipeline and SNR using high dynamic range technique (HDR), and evaluate benefits of such technique on polarimetric and spectral estimations,

Figure 8: Painting scene: (a) Visualization of Angle Of Linear Polarization for band 3. (b) Angle of polarization reveals brushstrokes direction. (c) Shapes of spectral reflectance for 3 pigments used in the painting.

2. Unify the polarimetric and spectral models using the dichromatic model, to get better understanding of spectral and polarimetric behaviors of specular and diffuse components,

• Training for spectral reflectance reconstruction is done using the Xrite ColorChecker. But reflectance estimation highly depends on the training dataset.

3. Data correlation analysis: could be investigated for spectral and polarization correlations among different materials.

L INKS The database and source code in Matlab are available at: https://github.com/pjlapray

Identified drawbacks:

• There is an energy balance disparity among all channels (see Figure 2d). Thus, all spectral bands don’t share the same noise level for a single shot.

R EFERENCE [1] R. S. Berns, L. A. Taplin, M. Nezamabadi, Y. Zhao, and M. Mohammadi. Practical spectral imaging using a color-filter array digital camera. Studies in Conservation, 2006.