Work of Camera Calibration (16)

I. Collecting experimental Results in Image Matching; preparing powerpoint for Meeting

II. Literature Review:

     - Patch Based Synthesis for Single Depth Image Super-Resolution (2012, ECCV) Gabriel J. Brostow

     - This purposed method can enhance the Depth information captured from Kinect camera: Originally, the depth map taken from a Kinect camera is not clear enough to specify objects within the monitoring area. Then this depth map will be useless for supporting the process of locating objects.

     - Applied this source code onto our collecting RGB-D maps:

Case 1

Depth map after applying normalization scheme onto each pixel-value

Depth map after applying Brostow's proposed method

III. Need to check which method is better for depth map enhancement. Brostow's method takes too long (...around 30 mins for a map under 640x480 size); yet, haven't check the difference of value between using 'the simple normalization method which is far more fast (...less than 1min for a map under 640x480 size)' and 'the Brostow's proposed one'.

Work of Camera Calibration (15)

I. Literature review in using RGB-D camera as well as Depth map:

     1. Walterio Mayol-Cuevas Research

     2. Egocentric Real-time Workspace Monitoring using an RGB-D Camera. (2012, IROS) Bristol Robtics Lab

     3. Detecting and Localising Multiple 3D Objects: A Fast and Scalable Approach. (2011, IROS-ASP-AVS) Bristol Robtics Lab

     4. Real-time Learning and Detection of 3D Texture-less Objects: A Scalable Approach. (2012, BMVC) Bristol Robtics Lab

II. Collecting RGB-D maps from Kinect

     - Trying to find the suitable distance from Kinect to Ojects
     - Preliminary testing of the ideas in mine and in the references

Work of Camera Calibration (14)

I. Adding 2 based way to improve the matching result:
 
   1. Deal with the 'n-to-1' cases: keep the most important one
   2. Make an attention area in both Kinect frame and hand-held camera frame:
       - keep the information which is 60% of the original frame (...just use the central 70% information)
       - Reason:
             a. A person's visual attention should be at the most central part of a view (...in hand-held frame)
             b. Information closing to the boundary is not important (... in Kinect frame)  
   3. Re-Confirming the metric for similarity computation:
       - Use SAD + NearestNeighborRatio
       - SURF features in 128-length extracting from each MSER feature location

   4. Part of results in scaling cases (...basic cases): number of matches are around 20 to 90

Scaling 1: before improve

Scaling 1: after improve

Scaling 2: before improve

Scaling 2: after improve

Scaling 5: before improve

Scaling 5: after improve

    5. Part of results in negative cases: number of matches are lower than 10

 II. Part of results in changing viewpoint: 
      -  number of matches has the similar trend as shown on the top 2 cases

III. Next... Considering to introduce Color and Depth information from the RGB-D camera, i.e. Kinect camera, seeing whether make something different in distinguishing position and negative cases

Work of Camera Calibration (13)

I. Run part of our datasets in order to confirm which setup is going to be taken for matching:

       - The final decision is:
              1. Use MSER descriptor to locate the possible location of useful feature:
                     - a region-based method
                     - set 'RegionAreaRange' to [5 RAR*0.3] in order to maintain global information
              2. Use SURF descriptor as a description for each feature generating from last step:
                     - a blob-based method
                     - using 128 as the length of a descriptor, which considers its orientation comparing to '64'
                     - Explanation of SURF in OpenCV
              3. Use 'SAD' and 'NearestNeighborSymmetric' metric to compute to confirm similarities among
                  Kinect frames and Hand-held camera frames.
              4. This is how the descriptor looks like:

       - Part of Results in doing matching under different scaling after improvement and modification the
          computation metrics as well as parameters (Right: Kinect frame; Left: Hand-held camera)

 
II. I would like to use some values such as 'SignOfLaplacian' and 'Orientation' in my modified 
     matching process. 

       - Current process in Matching is able to make the correspondence becoming much more precise and concentrate. Yet it is for describe how similar between a Kinect grames and a Hand-held camera frame. I need additional metrics to describe how different between these frames. (Right: Kinect frame; Left: Hand-held camera)

       - Eliminate 1-to-multiple relationship in the matching results... may need to sort out

       - Check whether the following information is useful or not:

III. Adaptive method to decide the threshold for filtering out inappropriate matches instead of 

       using a fix value

Work of Camera Calibration (12)

I. To locate the most valuable matches as the most important & correlated relationship

      - i.e. to keep the matches much more concentrated

Work of Camera Calibration (11)

I. Check every parameters in order to increase the number of MSER features
        - 'ThresholdDelta': decrease to 0.2
        - 'MaxAreaVariation': increase to 0.85
        - 'RegionAreaRange': change to [5  sizeOfImage*30%]

II. Check every parameters in order to increase the number of SURF features (in 128-bit)
        - 'MetricThreshold': decrease to 100
        - 'NumScaleLevels': increase to 6
        - 'NumOctaves': increase to 4

III. Check the 2 function calls to know what value is going to be feedback
        - 'extractFeatures': scale, signOfLaplacian, orientation, location, metric, count
        - 'detectMSERFeatures': count, location, axes, orientation, pixelList

IV. Check every parameters in 'Matching Function Call' in order to increase the number of matches
        - 'Method': change to 'NearestNeighborSymmetric'
        - 'MatchThreshold': increase to 10~20
        - 'Metric': change to 'SAD'
        - 'MaxRatio': 'Default'



Finally, we have 400~600 'MSER+SURF(in 128-bit)' features and 300~450 matches.

     

Summary of Weekly Meeting - (7)

1. Preparing Powerpoint for Weekly meeting; and trying to make a conclusion based on my current result; then giving a possible solution.

2. Summary of Weekly meeting:

    - Paying too much effort to achieve the final goal -> should substitute by using direct way

    - Giving a final tune of image matching in order to know I have done exhausted way to deal with that; then, we can go to the next phase in the individual project

    - Currently just stop for considering 3D matching stuff as well as build up a GUI for final presentation: don't go too far while doesn't close current step which has done an exhausted process for confirming that there is no way to improve the proposed method anymore.

Work of Camera Calibration (10)

1. Applying my work on our collecting dataset

2. Editing image stitching code (...maintaining in 2D space)