CS754 Assignment 1- Advanced Image Processing Solution

Submission instructions: You should ideally type out all the answers in Latex or else in MS Word with the equation editor. In either case, prepare a pdf file. Create a single zip or rar file containing the report, code and sample outputs and name it as follows: A1-IdNumberOfFirstStudent-IdNumberOfSecondStudent.zip. (If you are doing the assignment alone, the name of the zip file is A1-IdNumber.zip). Upload the file on moodle BEFORE 11:55 pm on 9th Feb. Beyond the cutoff time of 7 am on 10th Feb, no assignments will be accepted. Note that only one student per group should upload their work on moodle. Please preserve a copy of all your work until the end of the semester. If you have difficulties, please do not hesitate to seek help from me.
1. Let θ? be the result of the following minimization problem (BP): minkθk1 such that ky−ΦΨθk2 ≤ ε, where y is an m-element measurement vector, Φ is a m×n measurement matrix (m < n), Ψ is a n×n orthonormal basis in which n-element signal x has a sparse representation of the form x = Ψθ. Notice that y = Φx + η and ε is an upper bound on the magnitude of the noise vector η.
Theorem 3 we studied in class states the following: If√ Φ obeys the restricted isometry property with isometry

constant δ2s < 2 − 1, then we have kθ − θ?k2 ≤ C1s−1/2kθ − θsk1 + C2ε where C1 and C2 are functions of only δ2s and where ∀i ∈ S,θsi = θi;∀i /∈ S,θsi = 0. Here S is a set containing the s largest magnitude elements of θ.
A curious student asks the following questions: ‘(1) It appears that the upper bound on kθ−θ?k2 is reduced as s increases, which goes against the very premise of compressed sensing. How do we address this apparent discrepancy? (2) It also appears that the error bound is independent of m. How do you address this? (3) Now consider that I gave you another theorem (called Theorem 3A), which is the same as Theorem 3 except that it requires that δ2s < 0.3. Out of Theorem 3 and Theorem 3A, which is the more useful theorem? Why? (4) It appears that if I set ε = 0 in BP, I can always reduce the upper bound on the error even if the noise vector η has non-zero magnitude. What am I missing?’
Your job is to answer all four of the student’s questions. [5+5+5+5=20 points]
2. In class, we studied a video compressive sensing architecture from the paper ‘Video from a single exposure coded snapshot’ published in ICCV 2011 (See http://www.cs.columbia.edu/CAVE/projects/single_ shot_video/). Such a video camera acquires a ‘coded snapshot’ Eu in a single exposure time interval u. This coded snapshot is the superposition of the form where Ft is the image of the scene at instant t within the interval u and Ct is a randomly generated binary code at that time instant, which modulates Ft. Note that Eu, Ft and Ct are all 2D arrays. Also, the binary code generation as well as the final summation all occur within the hardware of the camera. Your task here is as follows:
(a) Read the ‘cars’ video in the homework folder in MATLAB using the ‘mmread’ function which has been provided in the homework folder and convert it to grayscale. Extract the first T = 3 frames of the video.
(b) Generate a H × W × T random code pattern whose elements lie in {0,1}. Compute a coded snapshot using the formula mentioned and add zero mean Gaussian random noise of standard deviation 2 to it. Display the coded snapshot in your report.
(c) Given the coded snapshot and assuming full knowledge of Ct for all t from 1 to T, your task is to estimate the original video sequence Ft. For this you should rewrite the aforementioned equation in the form Ax = b where x is an unknown vector (vectorized form of the video sequence). Mention clearly what A and b are, in your report.
(d) You should perform the reconstruction using Orthogonal Matching Pursuit (OMP). For computational efficiency, we will do this reconstruction patchwise. Write an equation of the form Ax = b where x represents the ith patch from the video and having size (say) 8×8×T and mention in your report what A and b stand for. For perform the reconstruction, assume that each 8 × 8 slice in the patch is sparse or compressible in the 2D-DCT basis. Carefully work out the error term in the OMP algorithm, and explain this in your report!
(e) Repeat the reconstruction for all overlapping patches and average across the overlapping pixels to yield the final reconstruction. Display the reconstruction and mention the relative mean squared error between reconstructed and original data, in your report as well as in the code.
(f) Repeat this exercise for T = 5,T = 7 and mention the mention the relative mean squared error between reconstructed and original data again.
(g) Note: To save time, extract a portion of about 120 × 240 around the lowermost car in the cars video and work entirely with it. In fact, you can show all your results just on this part. Some sample results are included in the homework folder.
(h) Repeat the experiment with any consecutive 5 frames of the ‘flame’ video from the homework folder. [35 points = 18 points for successful OMP implementation + 7 points for carefully presenting error term bound + 10 points for displaying of all results]
3. We will prove why the value of the coherence between m×n measurement matrix Φ (with all rows normalized
√ to unit magnitude) and n × n orthonormal representation matrix Ψ must lie within the range [1, n] (both
√
1 and n inclusive). Recall that the coherence is given by the formula µ(Φ,Ψ) = √nmaxi∈{0,1,...,m−1},j∈{0,1,...,n−1} itΨj|. Proving the upper bound should be very easy for you. |Φ
To prove the lower bound, proceed as follows. Consider a unit vector g ∈ Rn. We know that it can be ex-
pressed as g k as Ψ is an orthonormal basis. Now prove that .
Exploiting the fact that g is a unit vector, prove that the minimal value of coherence is attained when g k and that hence the minimal value of coherence is 1. [10 points]
4. Compressive sensing reconstructions involve estimating a sparse signal x 2 from a vector y ∈
) of compressed measurements of the form y = Φx where Φ ∈ Rm×n is the measurement matrix (assume there is no noise). Now answer the following questions, from first principles. Do not merely quote theorems or algorithms.
(a) If it is known that x has only 1 non-zero element and that the other elements are zero, can you uniquely estimate x if m = 1? If yes, how? If not, why not? Now further suppose, you knew beforehand the index (but not the value) of the non-zero element of x? Does this help you any further? If yes, how? If not, why not?
(b) If it is known that x has only 1 non-zero element and that the other elements are zero, can you uniquely estimate x if m = 2? If yes, how? If not, why not?
(c) If it is known that x has only 2 non-zero elements and that the other elements are zero, can you uniquely estimate x if m = 3? If yes, describe an algorithm that is guaranteed to estimate it accurately. If not, explain why not, and explain whether there are any special instances of Φ for which unique estimation is possible?
(d) Repeat part (c) with m = 4. [1+2+3+4=10 points]
(a) Explain the similarities and differences between this architecture and the video compressed sensing architecture using coded snapshots (Hitomi et al, ICCV 2011) that we studied in class.
(b) What cost function is the reconstruction algorithm in the paper based on? Explain the meaning and dimensions of every term in the cost function. (Note: you are not expected to describe the algorithms that minimize this cost function, but only mention the cost function itself) [8+7=15 points]
6. Here is our mandatory Google search question. Note that this is the only question for which you can perform a google search to get the answer. A very interesting application of compressed sensing is in the area of optical or electron microscopy. Your task is to search for a research paper which applies compressed sensing in this application. Answer the following questions briefly:
(a) Mention the title of the paper, where and when it was published and include a link to it.
(c) What reconstruction techhnique or cost function does the paper adopt for the sake of compressive reconstruction in this application? [3+4+4=10 points]