Assignment 2 Computational Photography Solution

1. Introduction




This assignment involves Texture Synthesis and Image Inpainting . The goal is to implement and evaluate the idea of Texture Synthesis by Non-Parametric Sampling proposed by Efros and Leung, ICCV99, and region filling by Criminisi et al “Region Filling and Object Removal by Exemplar-Based Image Inpainting”, TIP 2004




1.1. Paper and Code




Efros and Leung paper can be found at:




http://graphics.cs.cmu.edu/people/efros/research/EfrosLeung.html




A pseudo-code for their work is provided in:




http://graphics.cs.cmu.edu/people/efros/research/NPS/alg.html




The algorithm is also described in Forsyth and Ponce textbook in Chapter 6, and in Szeliski’s textbook in Section 10.5.




Criminisi’s paper can be found in http://research.microsoft.com/apps/pubs/default.aspx?id=70029







2. Main Task




2.1. Preliminaries




You will be given five images that you are going to use for synthesis.




Download the test images from Sakai resources




2.2. Implementation and Experiments




2.2.1. Texture Synthesis




Write a Python code, implementing Efros and Leung’s approach, to synthesize a 200x200 pixel image for each of the five example images T1,T2,…, T5.gif. Show synthesized images for T1,T2,…, T5.gif by changing the WindowSize parameter to 5, 9,11 pixels around the center.




2.2.2. Image Inpainting




Use your code (with modification if necessary) to fill in the gaps (black regions) in the images test_im1.bmp, test_im2.bmp. Try different parameter settings 5, 9, 11, and others if necessary, to see which one give you the best results. Comment about it in your report.


CS534
2/3



2.2.3. Object Removal




A better image inpainting approach was proposed in Criminisi, Perez and Toyama, “Region Filling and Object Removal by Exemplar-Based Image Inpainting”, Download and read that paper. Write your own implementation of the algorithm in Python.




The goal of this experiment is to compare Efros and Leung approach, which you implemented in 2.2.1, and this newer algorithm, for the task of object removal. The task is removing three objects/regions from the image test_im3 (the man on the left, the sign and pole on the bottom right, and the oversaturated area on the ground). Use and compare both approaches for this task. This is a color image, so first you need to convert it to a grayscale image. Run both codes on the image, with one of the regions marked to be filled at a time, and compare the results.




2.2.4. Image Quilting




In another paper by Efros and Freeman titled “Image Quilting for Texture Synthesis and Transfer”, SIGGRAPH 2001, another texture synthesis approach was proposed. Read that paper and download an implementation of that algorithm (or write your own) and compare its performance with Efros and Leung’s algorithm on the example images. Write a short essay (up to 1 page) summarizing the Imaging Quilting approach and describing in what ways it is similar or difference from Efros and Leung’s, and how they are compared quantitatively and/or qualitatively.







2.2.5. About Efficiency




It is very important to write an efficient code. To achieve that, avoid using for/while loops and try to use array and matrix operations instead, which are much faster. Also try to write your code using built-in functions, whenever possible. Report the running time.




2.2.6. Submitting the Code




You will submit all your code to Sakai. There should be at least two files; one for texture synthesis and one for image inpainting/region filing.




2.3. Report




Write a report with all necessary images properly labeled. Submit only a softcopy of your report to Sakai.




2.3.1. Images




Show synthesized images for T1,T2,…, T5.gif by changing the WindowSize parameter to 5, 9,11 pixels around the center.




In total you will have 21 synthesized images (7 images x 3 parameter settings). If you find a different better parameter set for inpainting part, submit result of that as well and comment about it in the report.




Show the output of the two region filling approaches on the three regions in image test_im3

CS534
3/3



3. Grading




Texture Synthesis - This part will be up to 30% of your grade




Image Inpainting - This part will be up to 30% of your grade




Region Filling - This part will be up to 25% of your grade




Image Quilting - This part will be up to 15% of your grade Grade Penalties




10% for late submission up to two days.




Up to 10% if the code is not running for some reason and/or giving significantly different results than the ones reported. Do not forget to submit all the files you have written and any external files that you have used. Create a ‘how to run’ section in your report if your code needs a special procedure to run.




Up to 50% if group work and/or code sharing is noticed. See Academic Integrity section for details. Any code downloaded from the internet has to be acknowledged in the report.







3.1. Academic Integrity




This is an individual assignment and it has to be completed by each person independently and not with groups. Sharing code or/and specific ideas is not allowed. The code, results and report that you have submitted will be compared with other people’s submissions and your grade will be severely affected if a clear resemblance exists. Instead of consulting to your classmates, please contact the TA or the Professor to clear out any confusing points about the assignment. You are expected to write your own code unless otherwise instructed, in such case you need to specify from where it was downloaded.




Please note that posting your code for the assignment in the future online is a violation of Rutgers policy since it might facilitate cheating in the future.