3D Reconstruction using Gray Code Illumination

Introduction

Gray code illumination (GCI) is one of the many techniques in rendering a 3D reconstruction of an object. It is a non-invasive technique, where only images of object are taken, under different pattern projection conditions. Another advantage is that it is insensitive to ambient illumination.
GCI works on the idea that patterns illuminated against a flat background will be distorted by objects placed against the background. These distortions are the cues to the shape of the object.

Gray code patterns

The patterns projected are a series of black and white stripes in the Gray code pattern. Each pattern has a bit weight, which is multiplied to the pattern itself. When these weighted patterns are summed, we now have what is called the bit plane stack (BPS). Each column in the BPS has a unique number. The distortion of this BPS due to the object is what we will keep track. Without the object, we have a background bit plane stack (BBPS). With the object, we have the OBPS.

Figure 1. Making the bit plane stack by summing the gray code patterns multiplied by the corresponding bit weights.

Methodology

Gray code patterns (Figure 2) are projected against a white background. Their images are taken, which will serve as the references for the distortion. The 3D object to be reconstructed is placed against the same white background. Several images of black and white patterns in the Gray Code configuration is projected onto the 3D object. For each gray code pattern, two images are taken. One is for when the object is present, and one is for background, in the absence of the object.

Figure 2. Object and background illuminated by different gray code patterns.

The images are then binarized. Each binarized image (from the background and object) is multipled by its corresponding bit weight, and added to make the background bit plane stack and object bit plane stack (Figure 3).

Figure 3. Left: Object bit plane stack. Right: Background bit plane stack

In the bit plane stacks, each stripe or color is a unique number. The shape of the object manifests in the distortion of the object bit plane stack. For each color, the displacement of the stripe is taken by subtracting the right-side edges. The differences are now the height of the object. The resulting 3D surface is plotted below (before and after median filtering).

Results

Reconstruction of the 3D surface 3 levels, which correspond to the levels of the pyramid. However, noise is still present, seen as sharp peaks in the height reconstruction (Figure 4a). This is remedied by applying median filtering. The resulting surface reconstruction is now smoother (Figure 4b).

The heights of each level are not the same, with the lowest level being the thickest.

(a) (b)

Figure 4. Left: Reconstruction before median filtering. Right: Reconstruction after median filtering.

To summarize, different patterns in Gray code configurations were used to successfully reconstruct a 3D surface. Several noise filtering methods were applied in the intermediate steps of the whole process. I give myself 10 points for this activity.

Many many thanks to Kaye for guidance in implementing the algorithms for this activity.

Camera Calibration using MATLAB toolbox

Camera Calibration

The camera calibration toolbox of MATLAB downloaded from http://www.vision.caltech.edu/bouguetj/calib_doc/ is an easier was of calibrating the camera.

It has an interface, which makes things easier.

But first several images of the grid oriented in different directions were taken. Shown below are the 16 images of the grid. In each image, the camera is fixed and the grid is moved.

Figure 1. Grid images.

The images in the folder must be named in sequence, for example DSC1.jpg, DSC2.jpg, DSC3.jpg, and so on. To input the images, click the button [Image names].

Figure 2. Calibration toolbox interface

The images are read by the program. The corners of the grid are selected one by one. A window showing the grid is displayed. The inner corners of the grid are selected, because the outer corners might be damaged.

Figure 3. Corner selection window.

The corners are selected starting from the upper left, clockwise. The GUI will ask for the dimension of the boxes in the grid. For this grid, each box is 1 inch wide and 1 inch long.

The corners are then extracted by the toolbox (use [Extract grid corners] button), and are labeled in the image (Figure 4).

Figure 4. Corners are extracted.

This is done for the rest of the images. You must be careful not to make any mistake. Otherwise, you will have to do this all over again, starting from the image input process. (This is most frustrating when you are in the last few images)

After the corners are extracted for all the images (in this case 16), the camera can now be calibrated. Click the button [Calibration]. The intrinsic parameters of the camera is now calculated by the program. Shown in Figure 5 is the output of the program.

Figure 5. Intrinsic camera parameters.

We have now calculated for the intrinsic parameters of the camera, such as focal length, principal point, skew, etc.

The extrinsic parameters can also be measured (click [Show extrinsic] button). The camera centered view or fixed-camera view is displayed (Figure 6). Here, the different orientations of the grid as captured by the camera is projected.

Figure 6. Extrinsic parameter (camera-centered)

Figure 7 shows the extrinsic parameters (world-centered). Here, the different locations of the camera is projected, as seen by the grid. This is basically the same as in Figure 6, because location is relative.

Figure 7. Extrinsic parameter (world-centered).

The camera was calibrated using the Camera Calibration Toolbox in Matlab. For more detailed instructions, refer to this link: http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/example.html

I give myself 10 points for this activity. I was able to get the intrinsic parameters the camera using the toolbox.