CSCI3290- Assignment 2: Image Stitching Solved

1           Assignment description
Image stitching is a technique to combine a set of images into a larger image by registering, warping, resampling and blending them together. A popular application for image stitching is creation of panoramas. Generally speaking, there are two classes of methods for image stitching, direct methods and feature-based methods. In this assignment, we will implement a feature-based method. It contains three parts: i) feature detection and matching, ii) homography estimation and iii) blending.

2           Assignment details
2.1         Feature detection and matching
Given two input images, detect SIFT features from them. And then establish the feature correspondence between SIFT features in the two input images using Brute-Force matcher. In OpenCV, there is a tutorial for SIFT https://docs.opencv.org/3.4/da/df5/tutorial_py_sift_intro.html, and a tutorial for Brute-Force matcher https://docs.opencv.org/3.4/dc/dc3/tutorial_py_matcher.html.

 This part is corresponding to extract and match feature function in the provided skeleton code:

                      def extract and match  feature (img  1 ,              img  2 ,      ratio

”””

                           : param img 1 :        input      image 1

                          : param img 2 :        input      image 2

                        : param     r a t i o t e s t :      ratio      for     the              robustness
 test =0.7):

test
 
 
 
                         : return              list  pairs  matched  keypoints :            a     l i s t

[ [ [ p1x , p1y ] ,[ p2x , p2y ] ] ]

””” list  pairs  matched  keypoints = [ ] # to be completed . . . .

return list  pairs  matched  keypoints
of             pairs
of
matched
points :
You need to do the following things in this function:

1)   extract SIFT feature from image 1 and image 2,

2)   use a bruteforce search to find pairs of matched features: for each feature point in img 1, find its best matched feature point in img 2,

3)   apply ratio test to select the set of robust matched points.

2.2         Homography estimation
The 2D image transformations (translation, rotation, scale, affine and perspective) can be represented as homography. Therefore, we can estimate the best homography by the matched pairs of features. Also, we employ RANSAC algorithm to eliminate the ”bad” matches. This can make our program more robust. In this section,you need to implement the RANSAC algorithm to find a robust homography between two input images using the feature correspondence.

This part is corresponding to find homography ransac function in the provided skeleton code:

1

5 – Assignment 2: Image Stitching                                                                                                                                                       2

  def find  homography  ransac ( list  pairs matched keypoints , threshold ratio inliers =0.85, threshold reprojection  error =3, max num trial =1000): ”””

 : param list  pairs matched keypoints : a l i s t of pairs of matched points : [ [ [ p1x , p1y ] ,[ p2x , p2y ] ] , . . . ] : param t h r e s h o l dr a t i o i n l i e r s :

threshold on the ratio of i n l i e r s over the total number of samples , accept the estimated homography i f ratio is higher than the threshold : param threshold  reprojection  error :

 threshold of reprojection error (measured as euclidean distance , in pixels ) to determine whether a sample is i n l i e r or outlier : param max num trial :

the maximum number of t r i a l s to take sample and do testing to find the best homography matrix

             : return      best  H :     the     best        found homography        matrix

”””

best  H = None

           # to     be      completed      . . .

return best  H
2.3         Blending
Having the estimated homography, we can warp the second image to align with the first image and then blend them together to get a a single panorama image. For warping, we employ inverse warping and bilinear resampling.

This part is corresponding to warp  blend image function in the provided skeleton code:

def warp  blend image (img  1 , H,                 img  2 ):

”””

              : param img 1 :       the       original       f i r s t     image

             : param H:        estimated       homography

               : param img 2 :       the       original        second     image

             : return       img panorama :         resulting            panorama image

”””

img  panorama = None

# to be completed . . . return img  panorama
You need to do the following things in this function:

1)   warp image img 2 using the homography H to align it with image img 1 (using inverse warping and bilinear resampling),

2)   stitch image img 2 to image img 1 and apply average blending to blend the two images into a single panorama image.

5 – Assignment 2: Image Stitching