CS534-Assignment 2 Solved

CS534 – Introduction to Computer Vision

Assignment 2

1.       Introduction
This assignment involves Texture Synthesis and Image Inpainting . The goal is to implement and evaluate the idea of Texture Synthesis by Non-Parametric Sampling proposed by Efros and Leung, ICCV99, and region filling by Criminisi et al “Region Filling and Object Removal by Exemplar-Based Image Inpainting”, TIP 2004

1.1.     Paper and Code
Efros and Leung paper can be found at: http://graphics.cs.cmu.edu/people/efros/research/EfrosLeung.html A pseudo-code for their work is provided in: http://graphics.cs.cmu.edu/people/efros/research/NPS/alg.html

The algorithm is also described in Forsyth and Ponce textbook in Chapter 6, and in Szeliski’s textbook in Section 10.5.

Criminisi’s paper can be found in http://research.microsoft.com/apps/pubs/default.aspx?id=70029

2.       Main Task
2.1.     Preliminaries
You will be given five images that you are going to use for synthesis.

Download the test images from Sakai resources

2.2.     Implementation and Experiments
2.2.1.     Texture Synthesis
Write a Python code, implementing Efros and Leung’s approach, to synthesize a 200x200 pixel image for each of the five example images T1,T2,…, T5.gif. Show synthesized images for T1,T2,…, T5.gif by changing the WindowSize parameter to 5, 9,11 pixels around the center.

2.2.2.     Image Inpainting
Use your code (with modification if necessary) to fill in the gaps (black regions) in the images test_im1.bmp, test_im2.bmp. Try different parameter settings 5, 9, 11, and others if necessary, to see which one give you the best results. Comment about it in your report.

2.2.3.     Object Removal
A better image inpainting approach was proposed in Criminisi, Perez and Toyama, “Region Filling and Object Removal by Exemplar-Based Image Inpainting”, Download and read that paper. Write your own implementation of the algorithm in Python.

The goal of this experiment is to compare Efros and Leung approach, which you implemented in 2.2.1, and this newer algorithm, for the task of object removal. The task is removing three objects/regions from the image test_im3 (the man on the left, the sign and pole on the bottom right, and the oversaturated area on the ground). Use and compare both approaches for this task. This is a color image, so first you need to convert it to a grayscale image. Run both codes on the image, with one of the regions marked to be filled at a time, and compare the results.   

2.2.4.     Image Quilting
In another paper by Efros and Freeman titled “Image Quilting for Texture Synthesis and Transfer”, SIGGRAPH 2001, another texture synthesis approach was proposed. Read that paper and download an implementation of that algorithm (or write your own) and compare its performance with Efros and Leung’s algorithm on the example images. Write a short essay (up to 1 page) summarizing the Imaging Quilting approach and describing in what ways it is similar or difference from Efros and Leung’s, and how they are compared quantitatively and/or qualitatively.

2.2.5.     About Efficiency
It is very important to write an efficient code. To achieve that, avoid using for/while loops and try to use array and matrix operations instead, which are much faster. Also try to write your code using built-in functions, whenever possible. Report the running time.