A database of polarimetric and multispectral images in the visible and NIR regions P IERRE -J EAN L APRAY, L UC G ENDRE , A LBAN F OULONNEAU , AND L AURENT B IGUÉ IRIMAS, U NIVERSITÉ DE H AUTE -A LSACE , M ULHOUSE , F RANCE
I NTRODUCTION
I MAGING P IPELINE
Multi-band polarization imaging, by mean of analyzing spectral and polarimetric data simultaneously, is a good way to improve the the information recovered from a scene. This work presents a database of polarimetric and multispectral images that combine visible and near-infrared (NIR) information. An experimental setup is built around a dual-sensor camera.
ˆ −1 M
12 Raw Image Acquisitions
Linearization + FPN & PNRU corrections
Image Registration
Multispectral generation from dual-RGB reassembly
ˆ R
Spectral Calibration
Spectral Estimation
Polarimetric Calibration
Stokes Estimation
−1 ˆ Ai
D ATABASE
Sˆi
R ESULTS
C ONTRIBUTIONS 1. A practical imaging pipeline, including a calibration procedure, for recovering spatial, spectral and polarimetric data from a set of photographs, 2. A database of multispectral and polarimetric images of various materials in the visible and near- infrared part of the spectrum.
(a) Macbeth classic
(d) Painting
(b) Macbeth enhancement
(a) I0-blue filter
(b) I45-blue filter
(c) I90-blue filter
(d) I135-blue filter
(e) I0-yellow filter
(f) I45-yellow filter
(g) I90-yellow filter
(h) I135-yellow filter
(i) I0-NIR
(j) I45-NIR
(k) I90-NIR
(l) I135-NIR
(c) Glass
(e) Fake & real leaves
(f) Potery
Figure 5: 12 raw images for the "Glass" scene after registration.
E XPERIMENTAL S ETUP (g) Food
(h) Fabrics
(a) s0 band 1
(b) s0 band 2
(c) s0 band 3
(d) s0 band 4
(e) s0 band 5
(f) s0 band 6
(g) s1 band 1
(h) s1 band 2
(i) s1 band 3
(j) s1 band 4
(k) s1 band 5
(l) s1 band 6
(m) s2 band 1
(n) s2 band 2
(o) s2 band 3
(p) s2 band 4
(q) s2 band 5
(r) s2 band 6
(i) Liquid
(j) Knife
Figure 6: 6-band linear Stokes images s0 , s1 and s2 .
Figure 3: sRGB visualization of the scenes. sRGB are reconstructed from a linear transform of the 6-band multispectral images after polarization component removing.
(a) AOLP band 3
(a) Painting
(b) Fake & real leaves
(c) Potery
Figure 1: Experimental setup and optical path.
(b) DOLP band 3
(c) DOLP zoom
Figure 7: Glass scene: (a)-(b) Visualization of Angle and Degree Of Linear Polarization for band 3. (c) Potential defect detection application.
Figure 4: Simple RGB visualizations without any spectral and polarization processing.
F UTURE D IRECTIONS (a)
(c)
(b)
(d)
Figure 2: (a) RGB/NIR camera sensitivities. (b) Transmittance of the bandpass filters. (c) RGB reassembly using blue-green (bg) and yellow (y) filters. (d) Final camera responses.
• The spectral signature is recovered using an RGB/NIR camera plus a couple of bandpass filters. Multispectral images are constructed from the dual-RGB method [1]. • The polarimetric feature is achieved using rotating linear polarization filters in front of the camera at four different angles (0, 45, 90 and 135 degrees).
(a) AOLP band 6
(b) AOLP zoom and brushstrokes direction
(c) Reflectance estimation for 3 pigments
1. Improve pipeline and SNR using high dynamic range technique (HDR), and evaluate benefits of such technique on polarimetric and spectral estimations,
Figure 8: Painting scene: (a) Visualization of Angle Of Linear Polarization for band 3. (b) Angle of polarization reveals brushstrokes direction. (c) Shapes of spectral reflectance for 3 pigments used in the painting.
2. Unify the polarimetric and spectral models using the dichromatic model, to get better understanding of spectral and polarimetric behaviors of specular and diffuse components,
• Training for spectral reflectance reconstruction is done using the Xrite ColorChecker. But reflectance estimation highly depends on the training dataset.
3. Data correlation analysis: could be investigated for spectral and polarization correlations among different materials.
L INKS The database and source code in Matlab are available at: https://github.com/pjlapray
Identified drawbacks:
• There is an energy balance disparity among all channels (see Figure 2d). Thus, all spectral bands don’t share the same noise level for a single shot.
R EFERENCE [1] R. S. Berns, L. A. Taplin, M. Nezamabadi, Y. Zhao, and M. Mohammadi. Practical spectral imaging using a color-filter array digital camera. Studies in Conservation, 2006.