Digital Image Processing: Homework #1

General Instructions:

    1. Read Homework Guidelines and MATLAB Function Guidelines for the information about homework programming, write-up and submission.
    2. If you make any assumptions about a problem, please clearly state them in your report.

    3. You need to understand the USC policy on academic integrity and penalties for cheating and plagiarism. These rules will be strictly enforced.


Problem 1: Image Demosaicing and Histogram Manipulation (45%)

(a) Bilinear Demosaicing (10%)

To capture color images, digital camera sensors are usually arranged in form of a color filter array (CFA), called the Bayer array, as shown in Figure 1. Since each sensor at a pixel location only captures one of the three primary colors (R, G, B), the other two colors have to be re-constructed based on their neighbor pixel values to obtain the full color. Demosaicing is the process of translating this Bayer array of primary colors into a color image that contains the R, G, B values at each pixel.





















Figure 1: (a) Single CCD sensor covered by a CFA and (b) Bayer pattern [1].

Implement the simplest demosaicing method based on bilinear interpolation. Exemplary demosaicing results are given in Figure 2. With this method, the missing color value at each pixel is approximated by bilinear interpolation using the average of its two or four adjacent pixels of the same color. To give an example, the missing blue and green values at pixel !,# are estimated as:

As for pixel    !,!, the blue and red values are calculated as:

Professor C.-C. Jay Kuo    Page 1 of 8
EE 569 Digital Image Processing: Homework #1



























(a)    (b)

Figure 2: (a) The original image and (b) the demosaiced image by bilinear interpolation.




























Figure 3: The House image before demosaicing.

    (1) Apply the bilinear demosaicing to the House image in Figure 3 and show your results.

    (2) Compare your demosaiced image with the color image House_ori which is obtained from a more advanced demosaicing algorithm. Do you observe any artifacts? If yes, explain the cause of the artifacts and provide your ideas to improve the demosaicing performance.

Professor C.-C. Jay Kuo    Page 2 of 8
        EE 569 Digital Image Processing: Homework #1


    (b) Histogram Manipulation (20%)

        ◦ Method A: the transfer-function-based histogram equalization method,

        ◦ Method B: the cumulative-probability-based histogram equalization method to enhance the contrast of the Hat image in Figure 4 below.

    (1) Plot the histograms of the original image. The figure should have the intensity value as the x-axis and the number of pixels as the y-axis.
    (2) Apply Method A to the original image and show the enhanced image. Plot the transfer function.

    (3) Apply Method B to the original image and show the enhanced image. Plot the cumulative histograms before and after enhancement.
    (4) Discuss your observations on these two enhancement results. Which one do you think is better and why?
Note that MATLAB users CANNOT use functions from the Image Processing Toolbox except displaying function like imshow().













Figure 4: Hat image

(c) Contrast Limited Adaptive Histogram Equalization (15%)

Instead of histogram equalization using global statistics from the entire image, adaptive histogram equalization breaks one image into several regions (tiles) and finds the histogram in each separately. Contrast Limited Adaptive Histogram Equalization (CLAHE) is one of the most popular algorithms. Please read the paper [2] carefully to learn about CLAHE.

In this problem, you will apply the CLAHE to do image haze removal, where you remove the fog in the provided image taken in a bad weather. Figure 5 shows an example before and after the haze removal. The process can be done through the following 3 steps.

Step-1: Transform the image from RGB color space to YUV color space. The transformation can be found in the Appendix.

Step-2: Apply a histogram equalization algorithm on the Y channel to get Y’.

Step-3: Combine Y’ with U and V, transform them back to RGB color space. The resulted image is your haze-removed image.

    (1) Explain CLAHE in your own words.

    (2) Perform the above three steps on Taj_Mahal.raw by applying the two histogram equalization methods in Problem 1(b) in step-2. Show the resulted images for both methods.

    (3) Repeat the three steps but apply the CLAHE in step-2. Here, you are allowed to use open-source code for CLAHE. For C++, you can use OpenCV. For Matlab, you can check function adapthisteq. Tune the

Professor C.-C. Jay Kuo    Page 3 of 8
EE 569 Digital Image Processing: Homework #1


hyperparameters including the number of tiles and the clip limit. Show the resulted image that you think is the most pleasant subjectively.

(4) Compare your results between (2) and (3). Discuss your observations.
















(a) Original foggy image    (b) Defogged image

Figure 5: An examples of image haze removal.


Problem 2: Image Denoising (40 %)

In this problem, you will implement a set of denoising algorithms to improve image quality. You can use the PSNR (peak-signal-to-noise-ratio) quality metric to assess the performance of your denoising algorithm. The PSNR value for R, G, B channels can be, respectively, calculated as follows:













Remove noise in the image in Figure 5(b), compare it with the original image in Figure 5(a), and answer the following questions:

    (a) Basic denoising methods (10%)

        (1) What is the type of embedded noise in Figure 5(b)? Justify your answer.

        (2) Apply a linear filter to the noisy image. Compare the performance of two choices of the filter parameters – the uniform weight function and the Gaussian weight function under different filter sizes.









Professor C.-C. Jay Kuo    Page 4 of 8
EE 569 Digital Image Processing: Homework #1




















(a) Original image    (b) Noisy image

Figure 5: The original and noisy Flower images.


(b) Bilateral Filtering (10%)

In most low-pass linear filters, we often see degradation of edges. However, using some nonlinear filters, we can preserve the edges. Bilateral filters are one such kind of filters. A discrete bilateral filter is given by:






where (  ,   ) is the neighboring pixel location within the window centered around (  ,   ), is the image with noise, is the filtered image. $ and % are two spread parameters.

        (1) Implement the bilateral denoising filter and apply it to the noisy image.

        (2) Explain the roles of $ and %. Discuss the change in filter's performance with respect to the values of $ and %.

        (3) Does this filter perform better than linear filters you implemented in Problem 2(a)? Justify your answer in words.

    (c) Non-Local Means (NLM) Filtering (10%)

The non-local mean filter utilizes the pixel value from a larger region rather the mean of a local window centered around the target pixel. A discrete non-local mean filter with Gaussian weighting function is as follows:










Professor C.-C. Jay Kuo    Page 5 of 8
EE 569 Digital Image Processing: Homework #1

where , are the noisy and filtered images respectively, &,' is the window centered around location (  ,   ), and ℎ is the filtering parameter, ( ≤ and ( ≤ denote the window size of your choice. The Gaussian weighted Euclidian distance between window   3  ),* 4and   3  +,,4 is defined as:
5  3  ),*4 −   3  +,,45-,.- =  8   .(  /, -)3  (   −  /, −  -) −   (   −  /, −  -)4
-
0!,0"∈ℵ


where ℵ denotes the local neighborhood centered at the origin, !, " ∈ ℵ denotes the relative position in the neighborhood window. > 0 is the standard deviation of the Gaussian kernel.

        (1) Apply the NLM filter (using any open-source code, e.g. C++ can use OpenCV, Matlab can check function imnlmfilt) to the noisy image. Try several filter parameters and discuss their effect on filtering process. Clearly state your final choice of parameters in your report.

(Note that there are four parameters to discuss: the big search window size ℵ, the small neighbor window size ( , the Gaussian smoothing parameter for the search window ℎ, the Gaussian smoothing parameter for the neighbor window . If the open source code doesn’t provide ways to adjust a certain parameter, just analyze that parameter theoretically.)

        (2) Compare the performance of NLM with filters used in Problem 2(a) and Problem 2(b).

    (d) Mixed noises in color image (10%)

Figure 6 (b) is a noisy color image corrupted with mixed types of noises. Please identify noise types in the image and answer the following questions:
    (1) What types of noises are there? Justify your answer.

    (2) What filters would you like use to remove mixed noise? Can you cascade these filters in any order? Justify your answer.
    (3) Get the best results in removing mixed noise. Include the following in your report:

        1. Describe your method and show its results

        2. Discuss its shortcomings.

        3. Give some suggestions to improve its performance.


















(a)    (b)

Figure 6: (a) the original Flower image (b) the Flower image with mixed noises.

Professor C.-C. Jay Kuo    Page 6 of 8
EE 569 Digital Image Processing: Homework #1


Problem 3: Special Effect Image Filters: Creating Frosted Glass Effect (15%)

An exemplary frosted glass effect for the Lake image is shown in Figure 8.

















Figure 8 An example of frosted glass effect

This effect can be created by simply replacing the color at each pixel with the color at a random pixel in its local neighborhood of size NxN. For example, considering N=3, the color of the center pixel at location 0 can be replaced by the color at any of the location 0-8 (including the location 0 which means unchanged).










    (1) Implement the frosted glass filtering with N = 5 or 7 and apply it to the Flower.raw image. Note that you can still formulate the process as a 2-D image filtering problem.

    (2) Repeat the process for the noisy image Flower_noisy.raw. Describe your observations and discuss whether the noise leads to any difference.
    (3) Considering the Bilateral filtering denoising algorithm. Try the following two processes:

        a. First, perform denoising on Flower_gray_noisy.raw. Then, apply the frosted-glass filtering.

        b. First, apply the frosted-glass filtering on Flower_gray_noisy.raw. Then, perform denoising. Compare your results and discuss your observations.















Professor C.-C. Jay Kuo    Page 7 of 8


EE 569 Digital Image Processing: Homework #1

Appendix:



Problem 1: Image Demosaicing and Histogram Manipulation

House.raw
768x512
8-bit
gray
House_ori.raw
768x512
24-bit
color(RGB)
Hat.raw
256x256
8-bit
gray
Taj_Mahal.raw
600x400
24-bit
color(RGB)
Problem 2: Image Denoising & Problem 3


Flower.raw
768x512
24-bit
color(RGB)
Flower_noisy.raw
768x512
24-bit
color(RGB)
Flower_gray.raw
768x512
8-bit
gray
Flower_gray_noisy.raw   768x512
8-bit
gray
Note: “768x512” means “width=768, height=512”.



Convert from RGB to YUV color space

Y = (0.257 * R) + (0.504 * G) + (0.098 * B) + 16

U = -(0.148 * R) - (0.291 * G) + (0.439 * B) + 128
V = (0.439 * R) - (0.368 * G) - (0.071 * B) + 128

Convert from YUV to RGB color space

R = 1.164(Y - 16) + 1.596(V - 128)

G = 1.164(Y - 16) - 0.813(V - 128) - 0.391(U - 128)
B = 1.164(Y - 16)    + 2.018(U - 128)

Reference Images

All images in this homework are from Google images [3], USC-SIPI image database [4], Kodak image dataset [5] or others [6].

References

    [1] M. E. Celebi et al. (eds.), Color Image and Video Enhancement.

    [2] Zuiderveld, Karel. “Contrast Limited Adaptive Histograph Equalization.” Graphic Gems IV. San Diego: Academic Press Professional, 1994. 474–485.
    [3] [Online] http://images.google.com/

    [4] [Online] http://sipi.usc.edu/database/

    [5] http://r0k.us/graphics/kodak/

    [6] https://comedytravelwriting.com/5-best-photos-of-the-taj-mahal-in-fog/















Professor C.-C. Jay Kuo    Page 8 of 8