Assignment 2: Image Stitching Solution

    • Assignment description

Image stitching is a technique to combine a set of images into a larger image by registering, warping, resampling and blending them together. A popular application for image stitching is creation of panoramas. Generally speaking, there are two classes of methods for image stitching, direct methods and feature-based methods. In this assignment, we will implement a feature-based method. It contains three parts: i) feature detection and matching, ii) homography estimation and iii) blending.

    • Assignment details

2.1    Feature detection and matching

Given two input images, detect SIFT features from them. And then establish the feature correspondence between SIFT features in the two input images using Brute-Force matcher. In OpenCV, there is a tutorial for SIFT https://docs.opencv.org/3.4/da/df5/tutorial_py_sift_intro.html, and a tutorial for Brute-Force matcher https://docs.opencv.org/3.4/dc/dc3/tutorial_py_matcher.html.

This part is corresponding to extract and match feature function in the provided skeleton code:


def  e x t r a c t
a n d m a t c h
f e a t u r e ( img 1 ,
img 2 ,
r a t i o  t e s t = 0 . 7 ) :
"""
















: param
img

1 :
i n p u t
image
1



: param
img

2 :
i n p u t
image
2



: param
r a t i o

t e s t :

r a t i o
f o r  t h e
r o b u s t n e s s
t e s t
: r e t u r n
l i s t

p a i r s

m a t c h e d

k e y p o i n t s :  a
l i s t
o f  p a i r s  o f  matched  p o i n t s :


    • [ [ p1x , p1y ] , [ p2x , p2y ] ] ]

"""

l i s t p a i r s m a t c h e d k e y p o i n t s =  [ ]


# t o  be  c o m p l e t e d    . . . .

return  l i s t p a i r s m a t c h e d k e y p o i n t s


You need to do the following things in this function:

    1) extract SIFT feature from image 1 and image 2,

    2) use a bruteforce search to nd pairs of matched features: for each feature point in img 1, nd its best matched feature point in img 2,

    3) apply ratio test to select the set of robust matched points.


2.2    Homography estimation

The 2D image transformations (translation, rotation, scale, a ne and perspective) can be represented as ho-mography. Therefore, we can estimate the best homography by the matched pairs of features. Also, we employ RANSAC algorithm to eliminate the "bad" matches. This can make our program more robust. In this sec-tion,you need to implement the RANSAC algorithm to nd a robust homography between two input images using the feature correspondence.

This part is corresponding to    nd homography ransac function in the provided skeleton code:





1

5 { Assignment 2: Image Stitching















2



























def f i n d h o m o g r a p h y
r a n s a c ( l i s t
p a i r s
m a t c h e d
k e y p o i n t s ,























t h r e s h o l d  r a t i o  i n l i e r s =0.85 ,























t h r e s h o l d r e p r o j e c t i o n
e r r o r =3,






















m a x n u m t r i a l =1000):






"""


































: param
l i s t



p a i r s






m a t c h e d

k e y p o i n t s :































a  l i s t
o f
p a i r s  o f
matched
p o i n t s :
[ [ [ p1x , p1y ] , [ p2x , p2y ] ] , . . . ]
: param
t h r e s h o l d


r a t i o

i n l i e r s :

























t h r e s h o l d
on






t h e  r a t i o
o f
i n l i e r s
o v e r  t h e
t o t a l
number
o f
samples ,
a c c e p t
t h e
e s t i m a t e d  homography  i f
r a t i o
i s  h i g h e r
than
t h e
t h r e s h o l d
: param
t h r e s h o l d





r e p r o j e c t i o n


e r r o r :



























t h r e s h o l d
o f






r e p r o j e c t i o n
e r r o r  ( measured
as  e u c l i d e a n
d i s t a n c e ,  i n
p i x e l s )


t o
d e t e r m i n e  w h e t h e r
a  sample
i s
i n l i e r
or
o u t l i e r

: param  m a x

n u m


t r i a l :
















t h e
maximum
number
o f  t r i a l s
t o  t a k e
sample
and
do
t e s t i n g
t o  f i n d
t h e
b e s t  homography
m a t r i x










: r e t u r n
b e s t

H :
t h e  b e s t  found
homography
m a t r i x















"""


































b e s t H = None



























# t o  be  c o m p l e t e d  . . .
















return
b e s t

H
































































2.3    Blending

Having the estimated homography, we can warp the second image to align with the rst image and then blend them together to get a a single panorama image. For warping, we employ inverse warping and bilinear resampling.

This part is corresponding to warp blend image function in the provided skeleton code:


def w a r p b l e n d

i m a g e ( img 1 ,  H,
img 2 ) :
"""








: param
img

1 :  t h e
o r i g i n a l
f i r s t
image
: param H:  e s t i m a t e d  homography

: param
img

2 :  t h e
o r i g i n a l
second  image
: r e t u r n
img

panorama :  r e s u l t i n g
panorama  image
"""









img panorama = None


# t o  be  c o m p l e t e d    . . .

return img panorama


You need to do the following things in this function:

    1) warp image img 2 using the homography H to align it with image img 1 (using inverse warping and bilinear resampling),

    2) stitch image img 2 to image img 1 and apply average blending to blend the two images into a single panorama image.

5 { Assignment 2: Image Stitching
3

    • Submission guidelines

3.1    Marks

    • extract and match feature: 40 points

    • nd homography ransac: 30 points

    • warp blend image: 30 points


3.2    Put personal information in your source code

In all your source    les, type your full name and student ID, just like:


#








# CSCI3290  Computational  Imaging  and  V i s i o n




#
D e c l a r a t i o n







# I  d e c l a r e  t h a t  t h e  a s s i g n m e n t  h e r e  s u b m i t t e d  i s  o r i g i n a l  e x c e p t  f o r  s o u r c e
#
m a t e r i a l  e x p l i c i t l y
a c k n o w l e d g e d .
I
a l s o  a c k n o w l e d g e
t h a t
I am
aware
o f
#
U n i v e r s i t y  p o l i c y  and  r e g u l a t i o n s
on
h o n e s t y  i n  academic
work ,
and  o f
t h e
#
d i s c i p l i n a r y  g u i d e l i n e s  and  p r o c e d u r e s  a p p l i c a b l e  t o
b r e a c h e s
o f  such
p o l i c y
#
and  r e g u l a t i o n s ,  as
c o n t a i n e d  i n
t h e  w e b s i t e




#
h t t p : / /www. cuhk . edu . hk / p o l i c y / a c a d e m i c h o n e s t y /




    • Assignment  2

    • Name  :

    • S t u d e n t  ID  :

    • Email  Addr  :

    • 


3.3    Late submission penalty

If you submit your solution after the due date, 10 marks will be deducted per day, and the maximal deduction is 30 marks even you delay more than 3 days. But there are hard deadlines as we need time to grade and submit grade. The hard deadline is 29 April 2020. After this day, we will not grade your assignment.