Merging of range images for inspection or safety applications James Mure-Dubois and Heinz H¨ugli University of Neuchˆ atel Institute of Microtechnology, 2000 Neuchˆ atel, Switzerland
Two- and Three-Dimensional Methods for Inspection and Metrology VI - 10.08.2008
Outline
1
Considered range imaging systems
2
Merging of range images
3
Applications
4
Conclusion
J.Mure-Dubois/ 10.08.2008
-2 -
Outline
1
Considered range imaging systems
2
Merging of range images
3
Applications
4
Conclusion
J.Mure-Dubois/ 10.08.2008
-3 -
3D microscope
Operation Field of view Typical accuracy Frame rate Range map size
J.Mure-Dubois/ 10.08.2008
Zoom 1× Zoom 10× 10 × 10 mm 1 × 1 mm 250 µm 25 µm 0.02 fps 0.04 fps 997 × 1016
-4 -
Time of flight camera network
Operation Field of view Typical accuracy Frame rate Range map size
J.Mure-Dubois/ 10.08.2008
Close range 1×1 m 50 mm 30 fps
Long range 5×5 m 350 mm 20 fps 176 × 144
-5 -
Motivation for range image merging
Application Main requirement Merging enables
3D microscope
TOF camera network
Inspection Accuracy
Safety systems Real-time capability
Field of view extension
Robustness to occlusion and field of view extension
Many approaches exist for range image registration and merging. We consider two specific systems, and investigate for each case the most suitable merging approach. J.Mure-Dubois/ 10.08.2008
-6 -
Outline
1
Considered range imaging systems
2
Merging of range images
3
Applications
4
Conclusion
J.Mure-Dubois/ 10.08.2008
-7 -
Merging of range images - Workflow 1
Convert range images to point clouds.
2
Express point clouds in same coordinates system.
3
Validate the merged results.
J.Mure-Dubois/ 10.08.2008
-8 -
From range map to point cloud - 3D microscope k =i ·N +j ∀ k = 0, 1, . . . , L L = (M · N) − 1
P = {p0 , p1 , . . . , pL }
x0 + i · m · ∆px pk = y0 + j · m · ∆py z0 + R(i, j) · ∆z
J.Mure-Dubois/ 10.08.2008
-9 -
From range map to point cloud - TOF camera k =i ·N +j ∀ k = 0, 1, . . . , L L = (M · N) − 1
P = {p0 , p1 , . . . , pL }
(i − cx ) · ∆px · zf (j − cy ) · ∆py · zf pk = f R(i, j) · √ 2 2
f +((i−cx )·∆px ) +((j−cy )·∆py )2
J.Mure-Dubois/ 10.08.2008
- 10 -
Coordinates transformation - 3D microscope
Point clouds must be merged to enable whole part inspection. For 3D microscope, the coordinates transformation is determined by sample displacement under the microscope (translation).
J.Mure-Dubois/ 10.08.2008
- 11 -
Coordinates transformation - TOF camera network
Range maps are acquired from different viewpoints. Merging requires to determine the 6 DOF coordinate transformation between the point clouds
J.Mure-Dubois/ 10.08.2008
- 12 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Planar primitives through RANSAC approach
A small, random set of points is used to define a planar primitive. The planar primitive is tested against the whole point cloud. Planar primitive can be used to help or validate alignment process. J.Mure-Dubois/ 10.08.2008
- 13 -
Outline
1
Considered range imaging systems
2
Merging of range images
3
Applications
4
Conclusion
J.Mure-Dubois/ 10.08.2008
- 14 -
Field of view increase - Caliper
Pm = P1
J.Mure-Dubois/ 10.08.2008
- 15 -
Field of view increase - Caliper
Pm = Pm ∪ P2
J.Mure-Dubois/ 10.08.2008
- 15 -
Field of view increase - Caliper
Pm = Pm ∪ P3
J.Mure-Dubois/ 10.08.2008
- 15 -
Field of view increase - Caliper
Pm = Pm ∪ P4
J.Mure-Dubois/ 10.08.2008
- 15 -
Field of view increase - Caliper
J.Mure-Dubois/ 10.08.2008
- 16 -
Field of view increase - Industrial part
J.Mure-Dubois/ 10.08.2008
- 17 -
Validation with planar primitives
J.Mure-Dubois/ 10.08.2008
- 18 -
TOF camera network alignment helped by planar primitives
J.Mure-Dubois/ 10.08.2008
- 19 -
TOF camera network alignment helped by planar primitives
J.Mure-Dubois/ 10.08.2008
- 19 -
TOF camera network alignment helped by planar primitives
J.Mure-Dubois/ 10.08.2008
- 19 -
Occlusion removal Single cam.
Front view
Top view
2 cameras
J.Mure-Dubois/ 10.08.2008
- 20 -
Occlusion removal Single cam.
Front view
Top view
2 cameras
J.Mure-Dubois/ 10.08.2008
- 20 -
Outline
1
Considered range imaging systems
2
Merging of range images
3
Applications
4
Conclusion
J.Mure-Dubois/ 10.08.2008
- 21 -
Conclusions Merging of range images was studied for two practical systems: 3D microscope and TOF camera network. For each system, a suitable, dedicated range image merging approach has been implemented. Alignment between views is crucial. Validity of alignment is confirmed for 3D microscope. Planar primitives extraction through RANSAC enables alignment for TOF camera networks. Possible improvements and extension: Use implicit connectivity information in range maps to produce geometric primitives. Extension: Automate 3D inspection process. J.Mure-Dubois/ 10.08.2008
- 22 -
End
Thank you for your attention !
J.Mure-Dubois/ 10.08.2008
- 23 -
