Estimate Homography to do 2D to 3D integration (5)

I. Assign thresholding method to the real tasks:

II. Pre-testing for checking the algorithm of mapping is workable between same images taken by the same machines (i.e., Kinect or Hand-held camera):

Kinect frame vs Kinect frame

Kinect frame vs Kinect frame without flip 

Hand-held camera img vs Hand-held camera img

III. Find an issue that the function 'fitgeotrans()' could not handle good results of affine transformation (... haven't found the reason yet); comparing to the former method, I try to modify the way of using another function 'maketform()', and being able to get the mapping result as I expect. 

   (1) The result of using 'fitgeotrans()':
   
     * Chose 12 points from 4 color markers (...3 points for each), which are 1 central points of each area and 2 points as its neighbours, to be the fixed points as the input.

Find 'Red Area': transformation between Kinect frame and Hand-held camera img

Find 'Yellow Area': transformation between Kinect frame and Hand-held camera img

   (2) The result of using 'maketform()' (... before modifying the choices of fixed points):

     * Chose points from red marker. Then compute 1 average position of this set of points and 2 points as its neighbours to be the fixed points as the input.

   

Find 'Red Area': transformation between Kinect frame and Hand-held camera img

Find 'Yellow Area': transformation between Kinect frame and Hand-held camera img

   (3) The result of using 'maketform()' (... after modifying the choices of fixed points):

      * Chose 9 points from 3 of 4 color markers (...3 points for each), which are 1 central points of each area and 2 points as its neighbours, to be the fixed points as the input.

Find 'Red Area': transformation between Kinect frame and Hand-held camera img

Find 'Yellow Area': transformation between Kinect frame and Hand-held camera img

IV. Finally, I choice to use the method: 

    (3) The result of using 'maketform()' (... after modifying the choices of fixed points):


V. Using 'The result of using 'maketform()' (... after modifying the choices of fixed points)' to do more testing:

Find 'Red Area': transformation between Kinect frame and Hand-held camera img

Find 'Yellow Area': transformation between Kinect frame and Hand-held camera img

VI. Some unconfirmed issues:

     1. What's the different between 'fitgeotrans()' & 'maketform()' (...minor)

     2. What's the reasonable way to chose the fixed points (... in number of 3 at most when using 'maketform()'; in unlimited number of points when using 'fitgeotrans()' )

     3. How to cope with the situation that fixed points are occluded by foreground object

     4. Any alternative way to improve or substitute current thresholding mechanism (... just set value of the R, G, B channels), which can be robust to illumination change (... find this issue when the viewpoint is changed)