Computer Vision PS1 Solution

Short answer problems [45 points]

Read through the provided Python, NumPy and Matplotlib introduction code and comments: http://cs231n.github.io/python-numpy-tutorial/ or https://filebox.ece.vt.edu/~F15ECE5554ECE4984/resources/numpy.pdf. Open an interactive session in Python and test the commands by typing them at the prompt. (Skip this step if you are already familiar with Python and NumPy.)

After reading the required documentation in the above section, to test your knowledge ensure that you know the outputs of the following commands without actually running them.

            ▪ import numpy as np

        (a) > x = np.random.permutation(1000)

        (b) > a = np.array([[11,22,33],[40,50,60],[77,88,99]])

            ▪ b = a[2,:]

        (c) > a = np.array([[11,22,33],[40,50,60],[77,88,99])

            ▪ b = a.reshape(-1)

        (d) > f = np.random.randn(5,1)

            ▪ g = f[f>0]

        (e) > x = np.zeros(10)+0.5

            ▪ y = 0.5*np.ones(len(x))

            ▪ z = x + y

        (f) > a = np.arange(1,100)

            ▪ b = a[::-1]

    1. Write a few lines of code to do each of the following. Copy and paste your code into the answer sheet. [20 points (5 each)]

1.1 Use numpy.random.rand to return the roll of a six-sided die over N trials.

1.2 Let y be the vector: y = np.array([11, 22, 33, 44, 55, 66]). Use the reshape command to

form a new matrix z that looks like this: [[11,22],[33,44],[55,66]]

1.3 Use the numpy.max and numpy.where functions to set x to the maximum value that occurs in z (above), and set r to the row number (0-indexed) it occurs in and c to the column number (0-indexed) it occurs in.

1.4 Let v be the vector: v = np.array([1, 4, 7, 1, 2, 6, 8, 1, 9]). Set a new variable x to be the number of 1’s in the vector v.

    2. Load the 100x100 matrix inputAPS1Q2.npy which is the matrix A. Fill the template functions in the script PS1Q2.py to load inputAPS1Q2.npy and perform each of the following actions on A. Submit the file PS1Q2.py. [25 points (5 each)]

2.1 Plot all the intensities in A, sorted in decreasing value. Provide the plot in your answer sheet. (Note, in this case we don’t care about the 2D structure of A, we only want to sort the list of all intensities.) To ensure consistency, you may use the gray colormap option.

2.2 Display a histogram of A’s intensities with 20 bins. Again, we do not care about the 2D structure.

Provide the histogram in your answer sheet.

2.3 Create and return a new matrix X that consists of the bottom left quadrant of A. (Use function prob_2_3 and return X.)

2.4 Create and return a new matrix Y, which is the same as A, but with A’s mean intensity value subtracted from each pixel. (Use function prob_2_4 and return Y.)

2.5 Create and return a new matrix Z that represents a color image the same size as A, but with 3 channels to represent R, G and B values. Set the values to be red (i.e., R = 1, G = 0, B = 0) wherever the intensity in A is greater than a threshold t = the average intensity in A, and black everywhere else. Save Z as outputZPS1Q2.png and be sure to view it in an image viewer to make sure it looks right. (Use function prob_2_5 and return Z.)

Short programming example [36 points]

The input color image inputPS1Q3.jpg has been provided. Fill the template functions in the script PS1Q3.py to perform the following transformations. Avoid using loops. Submit the file PS1Q3.py. Display all the re-sultant images in your answer sheet. [6 points each]

Note: In every part, the image that is returned from your function must have integer values in the range [0, 255] i.e uint8 format.

3.1 Load the input color image and swap its red and green color channels. Return the output image. (Use function prob_3_1 and return swapImg.)

3.2 Convert the input color image to a grayscale image. Return the grayscale image.

(Use function prob_3_2 and return grayImg.)

Perform each of the below transformations on the grayscale image produced in part 2 above. When plotting them, make sure you use the gray colormap option.

3.3 Convert the grayscale image to its negative image, in which the lightest values appear dark and vice versa. Return the negative image. (Use function prob_3_3 and return negativeImg.)

3.4 Map the grayscale image to its mirror image (flipping it left to right). Return the mirror image. (Use function prob_3_4 and return mirrorImg.)

3.5 Average the grayscale image with its mirror image (use typecasting). Return the averaged image. (Use function prob_3_5 and return avgImg.)

3.6 Create a matrix N whose size is same as the grayscale image, containing random numbers in the range [0, 255]. Save this matrix in a file called noise.npy. Add N to the grayscale image, then clip the resulting image to have a maximum value of 255. Return the clipped image. (Use function prob_3_6 and return addedNoiseImg.)

Be sure to submit noise.npy.

Tips: Do the necessary typecasting (uint8 and double) when working with or displaying the images. If you can’t find some functions in numpy (such as rgb2gray), you can write your own function. For example:

def rgb2gray(rgb):

return np.dot(rgb[...,:3], [0.2989, 0.5870, 0.1140])

Understanding Color [19 points]

4.1 The same color may look different under different lighting conditions. Images indoor.png and outdoor.png

are two photos of a same Rubik’s cube under different illuminances. Load the images and plot their R, G, B channels separately as grayscale images using plt.imshow() (beware of normalization). Then convert them into LAB color space using cv2.cvtColor() and plot the three channels again. Include the plots in your report. (Use function prob_4_1) [7 points]

4.2 In the section below, we have explained the process behind translating the cubical colorspace of RGB to the cylinder of hue, saturation, and value. Read through the explanation and fill in the template function in PS1Q4.py to load inputPS1Q4.jpg, convert the image from RGB to HSV and return the final HSV image. You may use for loops for this question. Submit the core logic and outputPS1Q4.jpg in the answer sheet. [12 points]

Note: To ensure consistency, do the necessary typecasting (double) to transform the image to the [0,1] scale before performing the below operations.

So far we’ve been focussing on RGB and grayscale images. But there are other colorspaces out there too we may want to play around with. Like Hue, Saturation, and Value (HSV).

Hue can be thought of as the base color of a pixel. Saturation is the intensity of the color compared to white (the least saturated color). The Value is the perception of brightness of a pixel compared to black. You can try out this demo to get a better feel for the differences between these two colorspaces.

Now, to be sure, there are lots of issues with this colorspace. But it’s still fun to play around with and relatively easy to implement. The easiest component to calculate is the Value, it’s just the largest of the 3 RGB components:

V = max(R; G; B)

Next we can calculate Saturation. This is a measure of how much color is in the pixel compared to neutral white/gray. Neutral colors have the same amount of each three color components, so to calculate saturation we see how far the color is from being even across each component. First we find the minimum value

m = min(R; G; B)

Then we see how far apart the min and max are:

C = V    m

and the Saturation will be the ratio between the difference and how large the max is:

S=C=V

Except if R, G, and B are all 0. Because then V would be 0 and we don’t want to divide by that, so just set the saturation 0 if that’s the case.

Finally, to calculate Hue we want to calculate how far around the color hexagon our target color is.





















Figure 1: Color Hexagon



4

We start counting at Red. Each step to a point on the hexagon counts as 1 unit distance. The distance between points is given by the relative ratios of the secondary colors. We can use the following formula from Wikipedia:

8
GCB


if V = R

>
undefined
C = 0
H0 =











B R




>





+ 2

if V = G




C




>







<









>
RCG +4

if V = B

>









>









:









H0


+ 1
if H’ < 0








H =
6







H0




(




otherwise


6




There is no "correct" Hue if C = 0 because all of the channels are equal so the color is a shade of gray, right in the center of the cylinder. However, for now let’s just set H = 0 if C = 0 because then your imple-mentation will match ours. (Use function prob_4_2 and return HSV.)



Deliverable Checklist

    1. 1.1-1.4 (answer sheet) code snippets for each question.

    2. 2.1-2.5 (code/files) PS1Q2.py, (answer sheet) corresponding plot for 2.1 & 2.2.

    3. 3.1-3.6 (code/files) PS1Q3.py, noise.npy, (answer sheet) 6 images.

    4. 4.1 (answer sheet) display the 2 plots (RGB, LAB)

    5. 4.2 (code/files) PS1Q4.py, (answer sheet) outputPS1Q4.png























This assignment is adapted from the following 3 sources:

PS0 assignment of Kristen Grauman’s CS 376: Computer Vision at UT Austin

HW1 assignment of David Fouhey’s EECS 442: Computer Vision at University of Michigan.

HW0 assignment of Joseph Redmon’s CSE 455: Computer Vision at University of Washington.


5