The Fundamental Matrix for Dipotric Cameras with Radial Distortion: Application on the Calibration of Wide Area Camera Networks
The epipolar geometry between two perspective views is a fundamental concept in computer vision. Due to the fact that any 3D point defines a single plane with the two camera centers, the projection of the point on the two views must satisfy a linear constraint. Since this relation is bilinear, it is usually described by a 3x3 fundamental matrix that can be estimated from a set of image correspondences. The fundamental matrix between two perspectives is broadly used because it encodes both the intrinsic parameters and the rigid displacement between cameras. The range of applications goes from calibration to 3D reconstruction, including motion estimation and features correspondence. However there is not a fundamental matrix for cameras with significant radial distortion. In this case the image formation is no longer described by a linear model, and the epipolar geometry can not be directly recovered from a set of correspondences. This fact severely limits the number of applications that can use cameras with wide-angle lens or low quality optics where the problem of radial distortion typically occurs.
<P>
We overcome these limitations by establishing for the first time a 4x4 fundamental matrix between views acquired by two cameras with radial distortion. This matrix encodes the distortion parameters, the camera intrinsics and the essential matrix. It is assumed that the lens distortion follows the first order division model. We show that the image formation is isomorphic to a projective mapping from a paraboloid to a plane, and that lines in the scene are projected into circles in the image. The bilinear constraint between two views is derived by embedding the projective plane into the 3D projective space. The 4x4 fundamental matrix is computed using the lifted coordinates of image correspondences. The algorithm to estimate the fundamental matrix is efficient and able to cope with outliers.
<P>
On the second part of the talk we will overview our current research on tele-immersion. Tele-immersion is a new paradigm for communication that associates 3D reconstruction with computer graphics, networking and media technologies. The objective is to create a medium for groups of people remote from each other to work and share experiences together in an immersive 3D virtual environment. We will present our toolbox to calibrate the wide area camera networks that are required for the data acquisition. The proposed approach is based on a factorization method that does not involve non-linear minimization. We will also give a brief account on the directions that are being pursued in order to develop algorithms able to generate photorealistic 3D reconstructions.
