Geometric Camera Parameters • What assumptions have we made so far? - All equations we have derived for far are written in the camera reference frames. - These equations are valid only when: (1) all distances are measured in the camera’s reference frame. (2) the image coordinates have their origin at the principal point.
- In general, the world and pixel coordinate systems are related by a set of physical parameters such as: * the focal length of the lens * the size of the pixels * the position of the principal point * the position and orientation of the camera
-2• Types of parameters (Trucco 2.4) - Two types of parameters need to be recovered in order for us to reconstruct the 3D structure of a scene from the pixel coordinates of its image points: Extrinsic camera parameters: the parameters that define the location and orientation of the camera reference frame with respect to a known world reference frame. Intrinsic camera parameters: the parameters necessary to link the pixel coordinates of an image point with the corresponding coordinates in the camera reference frame. Object Coordinates (3D)
World Coordinates (3D) extrinsic camera parameters Camera Coordinates (3D)
Image Plane Coordinates (2D)
Pixel Coordinates (2D, int)
intrinsic camera parameters
-3-
• Extrinsic camera parameters - These are the parameters that identify uniquely the transformation between the unknown camera reference frame and the known world reference frame. - Typically, determining these parameters means: (1) finding the translation vector between the relative positions of the origins of the two reference frames. (2) finding the rotation matrix that brings the corresponding axes of the two frames into alignment (i.e., onto each other)
-4- Using the extrinsic camera parameters, we can find the relation between the coordinates of a point P in world (P w ) and camera (P c ) coordinates: r 11 P c = R(P w − T ) where R = r 21 r 31
r 12 r 22 r 32
r 13 r 23 r 33
Xw Xc - If P c = Y c and P w = Y w , then Zw Zc X c r 11 Y = r c 21 Z c r 31
r 12 r 22 r 32
r 13 X w − T x r 23 Y w − T y r 33 Z w − T z
or X c = RT1 (P w − T ) Y c = RT2 (P w − T ) Z c = RT3 (P w − T ) where RTi corresponds to the i-th row of the rotation matrix
-5-
• Intrinsic camera parameters - These are the parameters that characterize the optical, geometric, and digital characteristics of the camera: (1) the perspective projection (focal length f ). (2) the transformation between image plane coordinates and pixel coordinates. (3) the geometric distortion introduced by the optics. From Camera Coordinates to Image Plane Coordinates - Apply perspective projection: Xc RT1 (P w − T ) x= f = f T , Zc R3 (P w − T )
Yc RT2 (P w − T ) = f T y= f Zc R3 (P w − T )
From Image Plane Coordinates to Pixel coordinates
x im
pixel frame
P
yim
Yc Camera Frame
y
Xc
Yw optical axis
Zc center of perspective projection
ox ,oy image plane frame
x
World Frame principal point
Xw Zw
x = − (x im − o x )s x or x im = − x/s x + o x y = − (y im − o y )s y or y im = − y/s y + o y where (o x , o y ) are the coordinates of the principal point (in pixels, e.g., o x = N /2, o y = M/2 if the principal point is the center of the image) and s x , s y correspond to the effective size of the pixels in the horizontal and vertical directions (in millimeters).
-6- Using matrix notation: x −1/s x im y im = 0 1 0
0 −1/s y 0
ox x oy y 1 1
Relating pixel coordinates to world coordinates RT1 (P w − T ) , −(x im − o x )s x = f T R3 (P w − T )
RT2 (P w − T ) −(y im − o y )s y = f T R3 (P w − T )
or x im
RT1 (P w − T ) = − fs x T + ox, R3 (P w − T )
y im
RT2 (P w − T ) = − fs y T + oy R3 (P w − T )
Image distortions due to optics Assuming radial distortion:
x = x d (1 + k 1 r 2 + k 2 r 4 ) y = y d (1 + k 1 r 2 + k 2 r 4 )
where (x d , y d ) are the coordinates of the distorted points (r 2 = x 2d + y 2d ) k 1 and k 2 are intrinsic parameters too but will not be considered here...
-7-
• Combine extrinsic with intrinsic camera parameters - The matrix containing the intrinsic camera parameters: − f /s x M in = 0 0
0 − f /s y 0
ox oy 1
- The matrix containing the extrinsic camera parameters:
M ex
r 11 = r 21 r 31
r 12 r 22 r 32
r 13 r 23 r 33
−RT1 T −RT2 T −RT3 T
- Using homogeneous coordinates: xh y = M M in ex h w
Xw m Xw Y w 11 Yw = m 21 =M Z Z w w m 31 1 1
m 12 m 22 m 32
m 13 m 23 m 33
Xw m 14 Yw m 24 Zw m 34 1
- Homogenization is needed to obtain the pixel coordinates: x im =
x h m 11 X w + m 12Y w + m 13 Z w + m 14 = m 31 X w + m 32Y w + m 33 Z w + m 34 w
y im =
y h m 21 X w + m 22Y w + m 23 Z w + m 24 = w m 31 X w + m 32Y w + m 33 Z w + m 34
- M is called the projection matrix (it is a 3 x 4 matrix). Note: the relation of 3D points and their 2D projections can be seen as a linear transformation from the projective space (X w , Y w , Z w , 1)T to the projective plane (x h , y h , w)T .
-8-
• The perspective camera model (using matrix notation) - Assuming o x = o y = 0 and s x = s y = 1 − fr 12 − fr 22 r 32
− fr 11 M p = − fr 21 r 31
− fr 13 − fr 23 r 33
fRT1 T fRT2 T −RT3 T
- Let’s verify the correctness of the above matrix: fRT1 T − fRT1 (P w − T ) Pw = − fRT2 (P w − T ) fRT2 T 1 T −RT3 T R3 (P w − T )
− fRT1 p = M p P w = − fRT2 T R3
- After homogenization (we get the same equations as in page 23): RT1 (P w − T ) x=− f T R3 (P w − T )
RT2 (P w − T ) y=− f T R3 (P w − T )
• The weak perspective camera model (using matrix notation) M wp
− fr 11 = − fr 21 0
− fr 12 − fr 22 0
− fr 13 − fr 23 0
fRT1 T fRT2 T RT3 (P − T )
where P is the centroid of the object (i.e., object’s average distance from the camera) - We can verify the correctness of the above matrix: − fRT1 p = M wp P w = − fRT2 0 0 0
fRT1 T − fRT1 (P w − T ) P fRT2 T w = − fRT2 (P w − T ) 1 T RT3 (P − T ) R (P − T ) 3
- After homogenization: x=− f
RT1 (P w − T ) RT3 (P − T )
y=− f
RT2 (P w − T ) RT3 (P − T )
-9-
• The affine camera model - The entries of the projection matrix are totally unconstrained: a11 M a = a21 0
a12 a22 0
a13 a23 0
a14 a24 a34
- The affine model does not appear to correspond to any physical camera. - Leads to simple equations and appealing geometric properties. - Does not preserve angles but does preserve parallelism.
