Multiple Views Geometry
                        Subhashis Banerjee
                Dept. Computer Science and Engineering
                           IIT Delhi
                     email: suban@cse.iitd.ac.in
                         January 12, 2008
                              1
          1 Epipolar geometry
          Fundamental geometric relationship between two perspective cameras:
          epipole: isthepointofintersectionofthelinejoiningtheopticalcenters-thebaseline
               - with the image plane. The epipole is the image in one camera of the optical
               center of the other camera.
          epipolar plane: is the plane defined by a 3D point and the optical centers. Or,
               equivalently, by an image point and the optical centers.
          epipolar line: is the line of intersection of the epipolar plane with the image plane.
               It is the image in one camera of a ray through the optical center and the image
               point in the other camera. All epipolar lines intersect at the epipole.
            Epipolar geometry provides a fundamental constraint for the correspon-
          dence problem
          1.1   Epipolar geometry: uncalibrated case
             ² Given the two cameras 1 and 2, we have the camera equations:
                                           ˜             ˜
                                      x =P Xandx =P X
                                       1    1       2     2
                             ² The optical center projects as
                                                                                                    ˜
                                                                                                   PX=0
                                                                                                      i
                             ² Writing
                                                                                            ˜
                                                                                           P =[P |−Pt]
                                                                                               i          i           i  i
                                 where P is 3×3 non-singular we have that t is the optical center.
                                                 i                                                                     i
                                                                                                             · t ¸
                                                                                      [P | −P t ]                  i     =0
                                                                                           i          i   i      1
                             ² The epipole e2 in the second image is the projection of the optical center of the
                                 first image:
                                                                                                             · t ¸
                                                                                                         ˜         1
                                                                                             e =P
                                                                                               2           2      1
                             ² The projection of point on infinity along the optical ray < t ;x > on to the
                                                                                                                                                     1      1
                                 second image is given by:
                                                                                            x =P P −1x
                                                                                               2          2 1           1
                             ² The epipolar line < e ;x > is given by the cross product e ×x .
                                                                          2     2                                                                 2         2
                             ² If [e ]          is the 3 × 3 antisymmetric matrix representing cross product with e ,
                                         2 ×                                                                                                                                       2
                                 then we have that the epipolar line is given by
                                                                                      [e ]      P P −1x =Fx
                                                                                         2 ×       2 1           1             1
                             ² Any point x on this epipolar line satisfies
                                                        2
                                                                                                x TFx =0
                                                                                                   2         1
                             ² F is called the fundamental matrix. It is of rank 2 and can be computed
                                 from 8 point correspondences.
                             ² Clearly Fe = 0 (degenerate epipolar line) and e TF = 0. The epipoles are
                                                     1                                                                        2
                                 obtained as the null spaces of F.
                          1.2             Epipolar geometry: calibrated case
                                  ² There are two camera coordinate systems related by R;T
                                                                                                                   ′
                                                                                                               X =RX+T
                                                                                                                                                                                                          ′
                                  ² Taking the vector product with T followed by the scalar product with X
                                                                                                               ′
                                                                                                          X·(T×RX)=0
                                                                                                                        ′     ′                     ′
                                       which expresses that vectors OX, O X and OO are coplanar.
                                  ² This can be written as
                                                                                                                      ′T
                                                                                                                  X EX=0
                                       where
                                                                                                                  E=[T] R
                                                                                                                                   ×
                                       is the Essential matrix.
                                  ² Image points and rays in Euclidean 3-space are related by:
                                                                             x                       X                           x′                     ′  X′ 
                                                                                                                                            ′                             ′
                                                                             y  = C Y  and  y =C  Y′ 
                                                                                  1                         Z                             1                           Z
                                  ² Hence, we have
                                                                                                            ′T      ′−T            −1
                                                                                                         x C EC x=0
                                  ² Thus, the relation between the essential and fundamental matrix is:
                                                                                                                            ′−T            −1
                                                                                                             F=C EC