During my Spring 2020 semester, I took the CS-635 class: Capturing and Rendering Real-World Scenes. This page presents one assignement I particularly liked to work on. We were asked to reconstruct a 3D scene from scratch.

Method

I used my phone to capture several images of a 3D scene I created. Some objects are easy to reconstruct (a small box), others are impossible to reconstruct (a Ferrero Rocher). The calibration pattern in the middle is of utmost importance because it is used to calibrate images, i.e., to find the 3D position of the camera. Here is one of the 16 images:

I used a very basic algorithm: first, I automatically calibrate all 16 images using the checkerboard pattern and estimate the poses of cameras. Then, I manually clicked feature points on each image. Each feature point is automatically reconstructed in the 3D scene using a linear system of equations and then a non-linear least square refinement. Then, I computed a 2D Delaunay triangulation of keypoints visible in the principal view (it’s possible to choose the principal view). Finally, I build a mesh out of the triangles and texture it with the image associated to the principal view. The whole 3D scene is displayed in an OpenGL view, with all original images displayed as transparent textures.

Result

More information

Link to the GitHub repository: CS635

Link to the class on Daniel Aliaga’s website: CS-635: Capturing and Rendering Real-World Scenes