$24
Computer Vision Assignment 2: Image Features Solved
• Harris Corner Detection (20 points)
Implement the Harris corner detector as described in class (lecture-5 slide-48 and tutorial-3) primarily using NumPy (you are prohibited from using OpenCV or Scikit-Image Harris corner detector function) (10 points).
Note that you should convert all images to grayscale for this question.
Harris corner detector has the following steps:
1. Compute image derivatives using sobel operator (optionally, blur rst).
2. Compute square of derivatives.
3. Apply gaussian ltering on the output of step-2.
4. Get cornerness function response on the output of step-3. (Determinant(H)-kTrace(H)2), where k=0.05. (You can vary value of k for your application).
5. Perform non-maxima suppression on the output of step-4 (the cornerness function ).
6. Threshold output of Non-maxima suppression at a user de ned threshold
(T).
You will apply Harris Corner Detector for three di erent images:
1. Checkerboard image Figure:1(a) (Input image). Change the threshold
value to get detected corners similar to Figure:1(a) (Harris Corner). Ob-serve and report the e ect of changing threshold values. Show the detected corners for at least three di erent threshold values (T). (3 points)
2. Shape image Figure:1(b). Try di erent values of thresholds and report your observations. Show detected corners for at least three di erent threshold values. (3 points)
2
Figure 2: Reference image of a book.
3. Take any one face image from Google Face thumbnail collection dataset. Apply the Harris corner detector on this image. Try di erent values of thresholds and report your observations. Show the detected corners for at least four di erent threshold values. (4 points)
• Invariance of SIFT Features (26 Points)
You are given a reference image of a book as shown in Figure:2. Verify the invariance of SIFT features under changes in image scale and rotation. For this question, you are allowed to use in-built OpenCV/Scikit-Image functions.
2.1 Invariance Under Scale
1. Compute SIFT keypoints for the reference image. (2 points)
2. Scale reference image using scaling factors of (0.2, 0.8, 2, 5). This should result in a total of 4 di erent transformed images. Display scaled images. (1 points)
3. Compute SIFT keypoints for all (total 4) transformed images. (2 points)
4. Match all keypoints of the reference image to the transformed images using a brute-force method. (2 points)
5. Sort matching key points according to the matching distance. (2 points)
3
6. Display the top ten matched keypoints for each pair of the reference image and a transformed image. (2 points)
7. Plot the matching distance for the top 100 matched keypoints. Plot indices of keypoints on the x-axis and corresponding matching distance on the y-axis. (1 points)
8. Discuss the trend in the plotted results. What is the e ect of increasing the scale on the matching distance? Reason the cause. (1 points)
2.2 Invariance Under Rotation
1. Compute SIFT keypoints for the reference image. (2 points)
2. Rotate reference image at the angle of (10, 90, 130, 180). Display rotated images. (1 points)
3. Compute SIFT keypoints for all (total 4) transformed images. (2 points)
4. Match all keypoints of the reference image to the transformed images using a brute-force method. (2 points)
5. Sort matching keypoints according to the matching distance. (2 points)
6. Display top ten matched keypoints for each pair of the reference image and a transformed image. (2 points)
7. Plot the matching distance for the top 100 matched keypoints. Plot indices of keypoints on the x-axis and corresponding matching distance on the y-axis. (1 points)
8. Discuss the trend in the plotted results. What is the e ect of increasing the angle of rotation on the matching distance? Reason the cause. (1 points)
(a) (b) (c)
Figure 3: Three di erent views of the same scene (a) 1Montreal (b) 2Montreal (c) 3Montreal [source]
4
• Image Stitching (30 Points)
You are given three di erent views of the same scene in a folder ‘stitching images’ (Figure:3). Follow these steps in order to stitch given images:
1. Compute SIFT keypoints and corresponding descriptors for images 1Mon-treal and 2Montreal. (2 points)
2. Find matching keypoints in 1Montreal and 2Montreal images and display the 20 best pairs. (4 points)
3. Find homography using the RANSAC method and apply the transfor-mation to 1Montreal. Image 2Montreal should not be transformed. (5 points)
4. Stitch transformed 1Montreal and the original 2Montreal together using linear image blending. Let us call this image 12Montreal. Display this image. (4 points)
5. Compute SIFT keypoints and corresponding descriptors for images 12Mon-treal and 3Montreal. (2 points)
6. Find matching keypoints in 12Montreal and 3Montreal images and display the 20 best pairs. (4 points)
7. Compute the homography using the RANSAC method. Apply the trans-formation to 3Montreal. Image 12Montreal should not be transformed. (5 points)
8. Stitch transformed 3Montreal and 12Montreal together using linear image blending. Display the resulting image. (4 points)
• Template Matching (14 Points)
In this problem, we will look at how to nd a template pattern in a cluttered scene. To make it more fun, we will use the Montreal Snow Festival ([source]) image { Figure:4(a), and you have to nd a person, shown in Figure:4(b).
1. Given a reference image (scene of Montreal Snow Festival) and a tem-plate face image, iterate over all pixel locations of the reference image and compare the local patch with the template image using the sum of square distance (SSD) metric. Implement this in NumPy (you are prohibited from using OpenCV function for this). (6 points)
2. Display SSD for the whole image. Find the location ((x, y) coordinate) where the SSD is minimum and see if you can nd the person there! (4 points)
3. Repeat the above process with the noisy template as shown in Figure:4(c). (4 points)
5
(b) (c)
(a)
Figure 4: (a) Cluttered scene of Montreal Snow Festival [source] (b) template face (c) noisy template face.
6
