In this project, filters are used as templates for finding simple patterns in the image. Correlation (or convolution, they are the same if the filter is symmetric) of an image with a filter can be viewed as comparing a template of what one wants to find against all local regions in the image. It is generally the case that the filter responds most strongly when it encounters an image pattern that looks like the filter. This is sometimes called matched filtering.
To compare template with a candidate region, normalized correlation is typically used to removed effects due to large, overall changes in brightness (see definition of normalized correlation in paper [1]). Use normalized correlation in this project. For all color images, you can first convert them to gray-scale images using rgb2gray().
Part A – Finding Stones
The following image displays Go (Wei Qi), a Chinese board game with white and black stones on the grid. Use template matching to find as many white and black stones in the image as you can. To create a stone template, you can crop one stone from the image.
(1) Apply template matching to find the stones in the image. Display the correlation map and the detected stones in a bounding box overlaid on the original image. Discuss the detection results.
(2) Discuss the conditions under which template matching works the best.
Part B – Finding Rock Paper Scissors
Freeman et al . in paper “Computer Vision for Interactive Computer Graphics” [1] introduced an interface where an open hand turns the television on. Read the paper, and in particular, the sections on “Small object tracking”.
Design a proof-of-concept experiment in which template matching is used to identify three hand shapes – rock, paper, or scissors, in a given image. Note, the templates and test images showing the hand shapes should come from different persons.
Reference
[1] Freeman, W., Anderson, D. and et al., “Computer Vision for Interactive Computer Graphics”, Computer Graphics and Applications, pp 42-53, 1998
