Starting from:
$30
CSCI5980-Assignment 4 Fundamental Matrix Solved
1 VLFeat Installation
One of key skills to learn in computer vision is the ability to use other open source code, which allow you not to re-invent the wheel. We will use VLFeat by A. Vedaldi and B. Fulkerson (2008) for SIFT extraction given your images. Install VLFeat from following:
http://www.vlfeat.org/install-matlab.html
Run vl demo sift basic to double check the installation is completed.
(NOTE) You will use this library only for SIFT feature extraction and its visualization. All following visualizations and algorithms must be done by your code.
2 SIFT Feature Extraction
Figure 1: Given your two cellphone images (left and right), you will extract SIFT descriptors and visualize them using VLFeat.
You will extract David Lowe’s SIFT (Scale Invariant Feature Transform) features from your cellphone images as shown in Figure 1. First, take a pair of pictures with your calibrated camera (intrinsic parameter, K, and radial distortion parameter, k, are precalibrated.) as follow:
1. Take the first picture and another one after moving one step right (1m).
2. Common 3D objects across different depths, e,g., buildings, ground plane, and tree, appear in both images.
3. Two images have to be similar enough so that the appearance based image matching can be applied, i.e., SIFT feature does not break, while maintaining the baseline between cameras (at least 1 m), i.e., similar camera orientation and sufficient translation.
4. Avoid a 3D scene dominated by one planar surface, e.g., looking at ground plane.
Write-up:
(SIFT visualization) Use VLFeat to visualize SIFT features with scale and orientation as shown in Figure 1. You may want to plot up to 500 feature points. You may want to follow the following tutorial:
http://www.vlfeat.org/overview/sift.html
3 SIFT Feature Matching
Figure 2: You will match points between I1 and I2 using SIFT features.
(NOTE) From this point, you cannot use any function provided by VLFeat.
The SIFT is composed of scale, orientation, and 128 dimensional local feature descriptor (integer), f ∈ Z128. You will use the SIFT features to match between two images, I1 and I2.
Write-up:
(1) (Nearest neighbor search) Let two sets of features be {f1,··· ,fN1} from I1 and {g1,··· ,gN2} from I2 where N1 and N2 are the number of features in image 1 and 2, respectively. Compute nearest neighbor per feature and visualize ({f1,··· ,fN1} → {g1,··· ,gN2} and {g1,··· ,gN2} → {f1,··· ,fN1}) as shown in Figure 2(a) and Figure 2(b). Note that the distance between two features is defined as d = kf − gk. You may use knnsearch function in MATLAB.
(2) (Ratio test) Filter out matches using the ratio test, i.e., keep the match if dij1/dij2 < 0.7 and discard otherwise, where dij1 and dij2 are the first and second nearest neighbors for the ith feature, respectively. Visualize the matches after the ratio test as shown Figure 2(d) and Figure 2(c).
(Bidirectional match) Visualize bidirectionally consistent matches as shown Figure 2(e). Compare the number of matches from (1) to (3).
One of key skills to learn in computer vision is the ability to use other open source code, which allow you not to re-invent the wheel. We will use VLFeat by A. Vedaldi and B. Fulkerson (2008) for SIFT extraction given your images. Install VLFeat from following:
http://www.vlfeat.org/install-matlab.html
Run vl demo sift basic to double check the installation is completed.
(NOTE) You will use this library only for SIFT feature extraction and its visualization. All following visualizations and algorithms must be done by your code.
2 SIFT Feature Extraction
Figure 1: Given your two cellphone images (left and right), you will extract SIFT descriptors and visualize them using VLFeat.
You will extract David Lowe’s SIFT (Scale Invariant Feature Transform) features from your cellphone images as shown in Figure 1. First, take a pair of pictures with your calibrated camera (intrinsic parameter, K, and radial distortion parameter, k, are precalibrated.) as follow:
1. Take the first picture and another one after moving one step right (1m).
2. Common 3D objects across different depths, e,g., buildings, ground plane, and tree, appear in both images.
3. Two images have to be similar enough so that the appearance based image matching can be applied, i.e., SIFT feature does not break, while maintaining the baseline between cameras (at least 1 m), i.e., similar camera orientation and sufficient translation.
4. Avoid a 3D scene dominated by one planar surface, e.g., looking at ground plane.
Write-up:
(SIFT visualization) Use VLFeat to visualize SIFT features with scale and orientation as shown in Figure 1. You may want to plot up to 500 feature points. You may want to follow the following tutorial:
http://www.vlfeat.org/overview/sift.html
3 SIFT Feature Matching
Figure 2: You will match points between I1 and I2 using SIFT features.
(NOTE) From this point, you cannot use any function provided by VLFeat.
The SIFT is composed of scale, orientation, and 128 dimensional local feature descriptor (integer), f ∈ Z128. You will use the SIFT features to match between two images, I1 and I2.
Write-up:
(1) (Nearest neighbor search) Let two sets of features be {f1,··· ,fN1} from I1 and {g1,··· ,gN2} from I2 where N1 and N2 are the number of features in image 1 and 2, respectively. Compute nearest neighbor per feature and visualize ({f1,··· ,fN1} → {g1,··· ,gN2} and {g1,··· ,gN2} → {f1,··· ,fN1}) as shown in Figure 2(a) and Figure 2(b). Note that the distance between two features is defined as d = kf − gk. You may use knnsearch function in MATLAB.
(2) (Ratio test) Filter out matches using the ratio test, i.e., keep the match if dij1/dij2 < 0.7 and discard otherwise, where dij1 and dij2 are the first and second nearest neighbors for the ith feature, respectively. Visualize the matches after the ratio test as shown Figure 2(d) and Figure 2(c).
(Bidirectional match) Visualize bidirectionally consistent matches as shown Figure 2(e). Compare the number of matches from (1) to (3).
1 file (6.2MB)
