CSCI3290-Assignment 2 Image Stitching Solved

Image stitching is a technique to combine a set of images into a larger image by registering, warping, resampling and blending them together. A popular application for image stitching is creation of panoramas. Generally speaking, there are two classes of methods for image stitching, direct methods and feature-based methods. In this assignment, we will implement a feature-based method. It contains three parts: i) feature detection and matching, ii) homography estimation and iii) blending.

1         Assignment details
1.1        Feature detection and matching
Given two input images, detect SIFT features from them. And then establish the feature correspondence between SIFT features in the two input images using Brute-Force matcher. In OpenCV, there is a tutorial for SIFT https://docs.opencv.org/3.4/da/df5/tutorial_py_sift_intro.html, and a tutorial for Brute-Force matcher https://docs.opencv.org/3.4/dc/dc3/tutorial_py_matcher.html.

This part is corresponding to extract and match feature function in the provided skeleton code:

                     def extract and match feature (img 1 ,             img 2 ,     ratio

”””

                         : param img 1 :       input      image 1

                         : param img 2 :       input      image 2

                      : param     r a t i o t e s t :      ratio      for     the            robustness
test =0.7):

test
 
 
 
                        : return            list pairs matched keypoints :           a     l i s t

[ [ [ p1x , p1y ] ,[ p2x , p2y ] ] ]

””” list pairs matched keypoints = [ ] # to be completed . . . .

return list pairs matched keypoints
of            pairs
of
matched
points :
You need to do the following things in this function:

1)   extract SIFT feature from image 1 and image 2,

2)   use a bruteforce search to find pairs of matched features: for each feature point in img 1, find its best matched feature point in img 2,

3)   apply ratio test to select the set of robust matched points.

1.2        Homography estimation
The 2D image transformations (translation, rotation, scale, affine and perspective) can be represented as homography. Therefore, we can estimate the best homography by the matched pairs of features. Also, we employ RANSAC algorithm to eliminate the ”bad” matches. This can make our program more robust. In this section,you need to implement the RANSAC algorithm to find a robust homography between two input images using the feature correspondence.

This part is corresponding to find homography ransac function in the provided skeleton code:

1

5 – Assignment 2: Image Stitching                                                                                                                  2

def find homography ransac ( list pairs matched keypoints , threshold ratio inliers =0.85, threshold reprojection error =3, max num trial =1000): ”””

: param list pairs matched keypoints : a l i s t of pairs of matched points : [ [ [ p1x , p1y ] ,[ p2x , p2y ] ] , . . . ] : param t h r e s h o l dr a t i o i n l i e r s :

threshold on the ratio of i n l i e r s over the total number of samples , accept the estimated homography i f ratio is higher than the threshold : param threshold reprojection error :

threshold of reprojection error (measured as euclidean distance , in pixels ) to determine whether a sample is i n l i e r or outlier : param max num trial :

the maximum number of t r i a l s to take sample and do testing to find the best homography matrix

            : return      best H :     the     best        found homography       matrix

”””

best H = None

           # to     be     completed       . . .

return best H
1.3        Blending
Having the estimated homography, we can warp the second image to align with the first image and then blend them together to get a a single panorama image. For warping, we employ inverse warping and bilinear resampling.

This part is corresponding to warp blend image function in the provided skeleton code:

def warp blend image (img 1 , H,                img 2 ):

”””

             : param img 1 :       the       original       f i r s t    image

            : param H:       estimated      homography

             : param img 2 :       the       original       second     image

            : return       img panorama :        resulting          panorama image

”””

img panorama = None

# to be completed . . . return img panorama
You need to do the following things in this function:

1)   warp image img 2 using the homography H to align it with image img 1 (using inverse warping and bilinear resampling),

2)   stitch image img 2 to image img 1 and apply average blending to blend the two images into a single panorama image.