Notes: Please use mycourses->Discussions for questions related to this assignment. You are free to discuss the questions with one another. However, the solutions that you submit must represent your own work. You are not permitted to copy code from each other, or from the internet. Present and discuss your results carefully and submit documented Matlab code that you have written to generate your results as a separate file. There are aspects of this assignment that are a bit open-ended, having to do with parameter choices to getting results that are qualitatively acceptable for the sequences we provide. This is something you can assess visually. Overall we will look not only at your implementations, but also how you present and discuss your results. Make sure that your figures and plots are in the PDF you submit for all questions.
For this question, you will be provided a starter code package. Please extract the package and follow the instructions in the README file before implementing anything.
In the Lucas-Kanade construction, the motion field is obtained by finding, for each pixel in an image, a corresponding pixel in the next frame of the sequence of images, in such a way that it minimizes the sum of squared intensity differences computed over a window. We saw in class in our discussion of 2D image registration that this leads directly to the use of the second moment matrix. In this assignment we will explore the use of this strategy for frame to frame optical flow (motion field estimation). Using essentially the same notation as in the notes, but treating In and In+1 as successive frames, the motion field (vx,vy) for frame In satisfies:
1
second moment matrix. To make this problem well posed, you will need to also blur the images with a 2D Gaussian of a certain σ prior to computing the gradient terms. This is because in Lucas-Kanade we use a first-order Taylor series approximation of the local intensity profile (we assume that locally the intensity varies linearly). Illustrate and discuss your results for a few frames in the Basketball and Backyard sequence, using the plotting functions we have provided. Discuss the performance and limitations of this one-shot algorithm.
2
