CV1- Assignment 2: Neighborhood Processing & Filters Solved

General guidelines

Your source code and report must be handed in together in a zip file (ID1 ID2 ID3.zip) before the deadline by sending it to Canvas Lab 2 Assignment. For full credit, make sure your report follows these guidelines:

•    Include an introduction and a conclusion to your report.

•    The maximum number of pages is 10 (single-column, including tables and figures). Please express your thoughts concisely. The number of words does not necessarily correlate with how well you understand the concepts. • Answer all given questions (in green boxes). Briefly describe what you implemented. Blue boxes are there to give you hints to answer questions. • Try to understand the problem as much as you can. When answering a question, give evidences (qualitative and/or quantitative results, references to papers, etc.) to support your arguments. • Analyze your results and discuss them, e.g. why algorithm A works better than algorithm B in a certain problem.

•    Tables and figures must be accompanied by a brief description. Do not forget to add a number, a title, and if applicable name and unit of variables in a table, name and unit of axes and legends in a figure.

 
 

 

1             Introduction
In this assignment, you will get familiar with fundamentals of neighborhood processing for image processing. These techniques allow for low-level image understanding via extraction of structural patterns such as edges and blobs. Similarly, they find an extensive use in image denoising and higher level image reasoning such as shape recognition. Moreover, neighborhood or block processing is one of the key components of Convolutional Neural Networks. Therefore, a good understanding of these procedures will be a stepping stone towards understanding more complex machinery used in computer vision and machine learning.

In subsequent sections of this assignment, we will first explain neighborhood processing and introduce low-level filters commonly used to analyze images. After that, we will see how these mathematical concepts relate to practice by working through fundamental tasks such as denoising and segmentation. By the end of this assignment, you will have an overall understanding of the following:

Gaussian and Gabor filters
Edge detection and image denoising
Texture-based image segmentation
2             Neighborhood Processing
Neighborhood processing is simply about looking around a point I(x,y) (i.e. pixel) in the image, I, and applying a function, h(k,l), which measures certain properties or relationships between the pixels in that localized window. The function, h(k,l), is generally referred to as the neighborhood operator or local operator. One of the most common forms of a neighborhood operator is a linear filter. Linear filters simply compute the weighted sum of neighboring pixel intensities and assign it to the pixel of interest (output Iout(i,j)). The filters in which we are interested here are usually represented as a square matrix.

Hint

Filters, kernels, weight matrices or masks are interchangeably used in the literature. A kernel is a matrix with which we describe a neighborhood operation. This operation can, for example, be edge detection or smoothing.

Linear filters are shifted over the entire image plane via operators such as correlation (⊗) and convolution (∗). Both of these operators are linear shift-invariant (LSI) implying that the filters behave the same way over the entire image. Discrete forms of these operators are given in the following:

We illustrate the overall idea of neighborhood processing in Figure 1.

Figure 1: The kernel or the mask convolves over the input image. In the case of linear filters, this is simply multiplying each pixel intensity with the corresponding weight in the kernel (see the yellowish 7x7 window where the kernel is placed). In the figure, the kernel is 7x7 averaging mask. You can see its effect by comparing the red (before filtering) and the green (after filtering) frames.

3             Low-level filters
In this section, you will design common linear filters used in neighborhood processing. We will focus in particular on Gaussian and Gabor filters.

3.1           Gaussian Filters
3.1.1          1D Gaussian Filter
The 1D Gaussian filter is defined as follows:

                                                               ,                                           (3)

where σ is the variance of the Gaussian. However, such formulation creates an infinitely large convolution kernel. In practice, the kernel is truncated with a kernel_size parameter such that , where b.c is the floor operator. As an example, if kernel_size equals 3, x ∈ {−1,0,1}.

3.1.2          2D Gaussian Filter
One of the most important properties of 2D Gaussian kernels is separability. Therefore, convolving an image with a 2D Gaussian is equivalent to convolving the image twice with a 1D Gaussian filter, once along the x-axis and once along the y-axis separately. A 2D Gaussian kernel can then be defined as a product of two 1D

Gaussian kernels:

                                                                                                                   )                                                 (4)

                                                                                                                                 )                                      (5)

3.1.3          Gaussian Derivatives
So far the Gaussian kernels that we computed are mainly targeted to image enhancement algorithms (e.g. denoising an image). These kernels can also be used for detecting changes in the image intensity pixels. These low-level features can then further be used as building blocks for more complicated tasks like object detection or segmentation.

Concretely, the first order derivative of the 1D Gaussian kernel is given by:

                                                                                                   (6)

Similarly, the first order derivative of the 2D Gaussian kernel can be obtained by computing ) and

3.2           Gabor filters
Gabor filters fall into the category of linear filters and are widely used for texture analysis. The reason why they are a good choice for texture analysis is that they localize well in the frequency spectrum (optimally bandlimited) and therefore work as flexible band-pass filters. See Figures 2 and 3.

3.2.1          1D Gabor Filters
For the sake of simplicity, we start by studying what a Gabor function is using 1D signals (e.g. speech). The idea will later be generalized to the 2D case, which is suited for our primary interest, images.

A Gabor function is a Gaussian function modulated with a complex sinusoidal carrier signal. Let us denote the Gaussian with x(t) and complex sinusoidal with m(t). Then, a Gabor function g(t) can be formulated by

                                                                              g(t) = x(t)m(t)                                                      (7)

where and m(t) = e−j2πfct = e−jwct. σ is the parameter determining the spread of the Gaussian and wc is the central frequency of the carrier signal.

Using Euler’s formula, we get the following:

                                                                                  )                                                                          (8)

(9)

                                                                                                                                        )]                             (10)

We can further arrange the terms and arrive at the following form
 
g(t) = ge(t) + jgo(t)
(11)
where ge(t) and go(t) are the even and odd parts arranged orthogonally on the complex plane Z2. In practice, one can use either the even or the odd part for filtering purposes (or one can use the complex form).

3.2.2          2D Gabor Filters
The Gabor filters can also be defined in 2D as well. The main difference lies in the dimensionality of the signals (i.e. carrier and gaussian). A sine wave in 2D is described by two orthogonal spatial frequencies u0 and v0 such that it is given as

s(x,y) = sin(2π(u0x+v0y)) where a 2D gaussian is simply with

C being a normalizing constant. 2D Gabor function then takes the following forms in the real and complex parts:

(12)

                                                                                                                                                                   ,           (13)