CSCI 363 Computational Vision--Fall 2016

    Assignments

    Home | | Schedule | | Assignments | | Lecture Notes

    Assignment 3, Due Friday, Oct 21

    In this assignment, we will explore the consequences of filtering images with different sizes of Gabor filters. We will also start working with fast Fourier transforms (fft's) and examine what happens to the frequency spectrum of images that are filtered with Gabor filters, similar to the receptive field organizations of simple cells in the striate cortex.

    Files needed for this assignment

    The M-Files and images that you need for this assignment are contained in the csci363/assignments/assign3 directory. You should already have copied the files into your assign3 directory during lab3. In MATLAB, set your Current Directory to the assign3 folder. Final submission details are given at the end of this handout.

    Viewing images in this assignment

    There are several points in this assignment where it is useful to view your images. You can view images in matlab using the function: pcolor. To view an image specified by the matrix, ImageMat, use the command: 

      pcolor(ImageMat)

    You will need to specify a gray-scale colormap with the command: 

      colormap(gray)
    Also, you can use the following commands to make your image look more like the one specified in the matrix: 

      axis square		%makes the vertical and horizontal dimensions equal
axis ij			%puts the image position 1, 1 at the top left corner
shading flat		%does not draw black grid lines between values
shading interp		%like shading flat, but better interpolation.  Slower, too

    To save you the trouble of typing these every time, I have provided you with a function, showPColor(imageMatrix) that calls pcolor(imageMatrix) and invokes all of the above commands. This function is in the showPColor.m file copied over with the assignment 3 files.

    Problem 1: Creating Gabor filters and filtering an image

    Lab3: Try to complete parts a, b and at least one fft in part c of this problem before leaving lab.

    In this problem, you will write a function to create a Gabor filters of different widths and frequencies. You will apply these filters to a real image using a 2D convolution and examine the results. You will also learn to use the fast fourier transform (fft2) function to examine the frequency components of the image, the gabor filters and the filtered image.

    a) Creating a Gabor filter
    First, create a gabor filter by the following steps:

    • Use the makeSin(freq, size) function to create a sinewave grating of frequency = 10 and size = 128:

      sinImage = makeSin(10, 128);

      This will create a 128 x 128 matrix of a sinewave values that has 10 cycles per image width. The sinewave oscillates along the horizontal axis only.

    • Use showPcolor(sinImage); to view the sinewave grating.
    • Use the makeGaussian(size, sigma) function to create a 128 x 128 matrix that has the values of a 2D Gaussian function with sigma value of 3:

      gaussian = makeGaussian(128, 3);

    • Open a new figure window (so you don't lose your sinewave image)
    • Use showPColor(gaussian); to view the Gaussian function.
    • Use component multiplication of the sinewave and Gaussian to create a Gabor filter:

      gaborFilter = sinImage .* gaussian;

      Note: The period before the asterisk is crucial!

    • Open a new image window before viewing your Gabor filter.
    • Use showPColor(gaborFilter); to view the Gabor filter.
    • Create a vertically oscillating Gabor filter by switching the rows and columns of your gaborFilter:

      verticalGabor = gaborFilter';

    • Open a new figure window.
    • Use showPColor(verticalGabor); to view the vertically oscillating Gabor filter.
    • Have Professor Royden check off your 4 images before you proceed.

    b) Write a function to create Gabor filters
    Use the function calls from part a to write a function, gabor = makeGabor(size, freq, sigma) that will create a Gabor filter with the given image size, frequency of the sinewave (in cycles per image width) and width of the Gaussian, sigma. Save your function as makeGabor.m. You will turn this function in with your assignment 3.

    Test your function out on several frequencies and sigma values. E.g. use frequency 20 and sigma 4 or frequency 5 and sigma 6. Use showPColor( ) to view the results. What happens when you increase the frequency of the sinewave? What happens when you increase the width of the Gaussian function?

    Have Professor Royden check off your makeGabor( ) function before you leave lab.

    c) Taking the Fourier Transform of a filter.

    MATLAB provides a function to approximage the 2D fourier transform of an image, using an algorithm known as "Fast Fourier Transform", or FFT. The function, fft2(image), results in a matrix whose values are the coefficients of the frequencies in the fourier spectrum of image. fft2 returns complex numbers for each frequency, but here we are interested only in the amplitude, so you can use the function abs to get the magnitude of the frequency function at each position. fft2 puts the zero frequency (DC component) in the upper left corner of the output matrix and the first frequency component (1 cycle per image) in the next position, and so on. The negative frequencies are in the right half of the output matrix (which is symmetric to the left half). To view the Fourier spectrum with the zero in the center of the image, use the function fftshift. Thus, to view the spectrum of sinImage, the command would be:

      fftImage = fftshift(abs(fft2(sinImage)));

    You can view the fft image using showPColor(fftImage). The zero frequency will be in the center of the image, positive horizontal frequencies are to the right of center and negative frequencies (the mirror image of positive frequencies) are to the left of center. Vertical frequencies are similarly above and below center.

    View the fft images of the following functions. For each type of function, answer the following questions:

    • What does the frequency distribution (i.e. image) for the fft of the function look like?
    • How does the frequency distribution change as you increase the frequency and/or sigma of the function?

    Functions to test:

    1. sinewave gratings of frequency 5, 10, 20
    2. Gaussian functions with sigma of 3 and 5
    3. Gabor functions, with the following (freq, sigma) pairs: (10, 3), (10, 5), (20, 3), (20, 5)
    4. A Vertical gabor function with freq 20, sigma 3.

    d) Filtering an image
    To filter an image with a Gabor filter, we take the 2D convolution of the image with the Gabor function. MATLAB provides a 2D convolutions function, conv2(matrix1, matrix2).

    Use the command:
    yacht = imread('yacht.jpg');
    to read in the yacht image. You will need to cast the type of data in yacht to double precision floating point when convolving it using conv2. Thus, your convolution commands will look like the following:

    yachtF5 = conv2(double(yacht), gabor5);

    Use the conv2( ) function to filter the yacht image with each of the following gabor functions:

    • Frequency = 5, sigma = 6
    • Frequency = 10, sigma = 3
    • Frequency = 20, sigma = 3
    • Vertical Gabor function with frequency = 20, sigma = 3
    Note that you created these in part c.

    For each of the filtered images, use showPColor to view them. Describe the images and how they compare to one another. What kind of edges and how much detail can you see?

    Finally, take the fft2 of each of the filtered images. View the frequency spectra of each using showPColor( ). How do the frequency spectra differ for the 4 filtered images? What does this tell you about the relationship between the frequencies contained in an image and the image itself?

    Problem 2: Plaids and Checkerboards

    a) Create plaid and checkerboard gratings

    Create a plaid grating by adding together your vertical sinewave with a horizontal sinewave:

    plaid = sinImage + sinImage';

    Use a sinImage with frequency 10 and size 128.

    Now create a checkerboard grating by multiplying the horizontal and vertical gratings:

    checkerboard = sinImage .* sinImage';

    Use showPColor to view each of the gratings.

    Now take the fft of each of the gratings you created. Use showPColor( ) to view the frequency spectrum of each.

    Answer the following questions:

    • How do the appearances of the two gratings differ?
    • How do the frequency spectra of the two gratings differ?

    b) Filtering the gratings

    Create a Gabor filter with frequency 10 and sigma 3, and use it to filter (convolve) each of the gratings from part a).

    Use showPColor( ) to view the filtered images.

    Take the fft of each of the filtered gratings and use showPColor( ) to view the frequency spectra of each.

    Answer the following questions:

    • How does the appearance of each of the filtered images differ from the original?
    • How do the frequency spectra of the filtered images differ from those of the originals?
    • Why does the filtering appear to have such different effects on the two kinds of plaids? (Hint: when you convolve in the spatial domain, you multiply the frequency spectra in the frequency domain. What happens when you multiply the frequency spectrum of the gabor filter with the spectra of each of the gratings?)

    Problem 3: Using Plaids and Checkerboards to Examine cells in V1

    DeValois and colleagues used plaids and checkerboards in an experiment to show that cells in V1 respond to spatial frequencies of a stimulus, rather than to the luminance edges. To do so, they measured the orientation tuning of simple and complex cells using either a drifting sinewave grating or a drifting checkerboard. They found that the orientation tuning of V1 cells when presented with a checkerboard differed by +/- 45 deg from the orientation tuning when presented with a sinewave grating. They used this as evidence that these cells are responding to the frequency components of the stimulus rather than to the luminance edges. Based on your knowledge of the frequency spectra of checkerboards, explain why this tuning difference between gratings and checkerboards indicates a response to frequencies rather than edges.

    The original paper describing this experiment is available on Moodle (DeValois Checkerboard paper). You may find it helpful to read some or all of the paper.

    Submission details: Hand in a hardcopy of your makeGabor.m code files and your answers to the questions for Problems 1-3. Please also email me an electronic copy of your code file by attaching it to an email to me at croyden@holycross.edu. Attach the code file to the message.

    Home | | Schedule | | Assignments | | Lecture Notes

    Constance Royden--croyden@mathcs.holycross.edu
    Computer Science 363--Computational Vision
    Last Modified: September 28, 2016
    Page Expires: Auguest 24, 2017