I. Homography Estimation:
- http://www.cs.ubc.ca/grads/resources/thesis/May09/Dubrofsky_Elan.pdf
- https://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0CD4QFjAB&url=http%3A%2F%2Fvims.cis.udel.edu%2F~chandra%2F689%2FSpring06%2Faffine%2Bhomography.ppt&ei=0dFoU_GbI-WW0QXjqIGoDw&usg=AFQjCNF0NHqgjF29AkfQL82_mZmc-NJhIQ&sig2=B7HooLURQEwXBDFI9LrkaQ
II. MATLAB Function of Homography:
1. The following method is for mapping one area to another. The transform matrix can be applied to the entire image for transforming each point from one Image to another.
- maketform: http://www.mathworks.co.uk/help/images/ref/maketform.html
- tformfwd: http://www.mathworks.co.uk/help/images/ref/tformfwd.html
- tforminv: http://www.mathworks.co.uk/help/images/ref/tforminv.html
2. This is an alternative way to maintain the homography of the entire image. (...which fits our needs to do mapping from one fixed point in hand-held camera to another moving point in the Kinect frame)
- fitgeotrans: http://www.mathworks.co.uk/help/images/ref/fitgeotrans.html#btxcsco-2
- fixed points: points of markers
moving points: points located on an object
Monday, 5 May 2014
Friday, 2 May 2014
Estimate Homography to do 2D to 3D integration (1)
I. Studying the Homography and search useful information
Thursday, 1 May 2014
Summary of Weekly Meeting - (8): Co-meeting with Matina
I. Memo for the 1st component:
- It is fine but not robust enough under rotation and viewpoint change:
try to use some rotation invariant descriptors to substitute the SURF descriptor (...SIFT is one of the common one to try)
- Don't care too much about the number of matches, the number of 'correct' matches is more important
- Don't worried about the speed of time, this is the big constraint for now, i.e., offline process (...So this is one of the reason that SURF descriptor can/should be substituted)
- Fine tune for this 1st component while I am available (... solve the issue in rotation invariant)
*Suggestion:
Keep going to the 2nd component, then back to 1st component to do some improvement. Time is limited, and current algorithm somehow is fine with a main issue of rotation invariant. Moreover, this step may take ages to try and test in order to find the best/acceptable solution.
II. Memo for the 2nd component:
- This is for 2 purposes:
1. Do 2D to 3D integration
2. As the most simple way to do matching under 'RST invariant'... which can be a ground truth method to improve/confirm the result and performance of my proposed method (i.e., the 1st component)
3. The Strength in using markers:
- The most common way in MIS
- To locate an object without considering its properties such as texture
- A group of markers which can circle out an object
- The group of markers can be moved to simulate Affine transformation
- Tools (...use one of them):
1. Color marks: red, yellow, blue, green (...patterns is not necessary)
2. A white paper with painted color markers: red, yellow, blue, green
3. Directly assign value into 2 matrix
4. Directly choose the information of location of points in 2 matrix
- Main steps:
1. Color segmentation
2. Estimate Homography (i.e., Affine Transformation): H = [R T]
3. Kindly note from Matina and George:
*Suggestion:
- Check the idea and method of homography in MATLAB / Source Code
- Check the book - The Bible of OpenGL
- Asking people
- It is fine but not robust enough under rotation and viewpoint change:
try to use some rotation invariant descriptors to substitute the SURF descriptor (...SIFT is one of the common one to try)
- Don't care too much about the number of matches, the number of 'correct' matches is more important
- Don't worried about the speed of time, this is the big constraint for now, i.e., offline process (...So this is one of the reason that SURF descriptor can/should be substituted)
- Fine tune for this 1st component while I am available (... solve the issue in rotation invariant)
*Suggestion:
Keep going to the 2nd component, then back to 1st component to do some improvement. Time is limited, and current algorithm somehow is fine with a main issue of rotation invariant. Moreover, this step may take ages to try and test in order to find the best/acceptable solution.
II. Memo for the 2nd component:
- This is for 2 purposes:
1. Do 2D to 3D integration
2. As the most simple way to do matching under 'RST invariant'... which can be a ground truth method to improve/confirm the result and performance of my proposed method (i.e., the 1st component)
3. The Strength in using markers:
- The most common way in MIS
- To locate an object without considering its properties such as texture
- A group of markers which can circle out an object
- The group of markers can be moved to simulate Affine transformation
- Tools (...use one of them):
1. Color marks: red, yellow, blue, green (...patterns is not necessary)
2. A white paper with painted color markers: red, yellow, blue, green
3. Directly assign value into 2 matrix
4. Directly choose the information of location of points in 2 matrix
- Main steps:
1. Color segmentation
2. Estimate Homography (i.e., Affine Transformation): H = [R T]
3. Kindly note from Matina and George:
- Check the idea and method of homography in MATLAB / Source Code
- Check the book - The Bible of OpenGL
- Asking people
Wednesday, 30 April 2014
Work of Camera Calibration (16)
I. Collecting experimental Results in Image Matching; preparing powerpoint for Meeting
II. Literature Review:
- Patch Based Synthesis for Single Depth Image Super-Resolution (2012, ECCV) Gabriel J. Brostow
- This purposed method can enhance the Depth information captured from Kinect camera: Originally, the depth map taken from a Kinect camera is not clear enough to specify objects within the monitoring area. Then this depth map will be useless for supporting the process of locating objects.
- Applied this source code onto our collecting RGB-D maps:
II. Literature Review:
- Patch Based Synthesis for Single Depth Image Super-Resolution (2012, ECCV) Gabriel J. Brostow
- This purposed method can enhance the Depth information captured from Kinect camera: Originally, the depth map taken from a Kinect camera is not clear enough to specify objects within the monitoring area. Then this depth map will be useless for supporting the process of locating objects.
- Applied this source code onto our collecting RGB-D maps:
![]() |
| Case 1 |
| Depth map after applying normalization scheme onto each pixel-value |
![]() |
| Depth map after applying Brostow's proposed method |
III. Need to check which method is better for depth map enhancement. Brostow's method takes too long (...around 30 mins for a map under 640x480 size); yet, haven't check the difference of value between using 'the simple normalization method which is far more fast (...less than 1min for a map under 640x480 size)' and 'the Brostow's proposed one'.
Tuesday, 29 April 2014
Work of Camera Calibration (15)
I. Literature review in using RGB-D camera as well as Depth map:
1. Walterio Mayol-Cuevas Research
2. Egocentric Real-time Workspace Monitoring using an RGB-D Camera. (2012, IROS) Bristol Robtics Lab
3. Detecting and Localising Multiple 3D Objects: A Fast and Scalable Approach. (2011, IROS-ASP-AVS) Bristol Robtics Lab
4. Real-time Learning and Detection of 3D Texture-less Objects: A Scalable Approach. (2012, BMVC) Bristol Robtics Lab
II. Collecting RGB-D maps from Kinect
- Trying to find the suitable distance from Kinect to Ojects
- Preliminary testing of the ideas in mine and in the references
1. Walterio Mayol-Cuevas Research
2. Egocentric Real-time Workspace Monitoring using an RGB-D Camera. (2012, IROS) Bristol Robtics Lab
3. Detecting and Localising Multiple 3D Objects: A Fast and Scalable Approach. (2011, IROS-ASP-AVS) Bristol Robtics Lab
4. Real-time Learning and Detection of 3D Texture-less Objects: A Scalable Approach. (2012, BMVC) Bristol Robtics Lab
II. Collecting RGB-D maps from Kinect
- Trying to find the suitable distance from Kinect to Ojects
- Preliminary testing of the ideas in mine and in the references
Monday, 28 April 2014
Work of Camera Calibration (14)
I. Adding 2 based way to improve the matching result:
1. Deal with the 'n-to-1' cases: keep the most important one
2. Make an attention area in both Kinect frame and hand-held camera frame:
- keep the information which is 60% of the original frame (...just use the central 70% information)
- Reason:
a. A person's visual attention should be at the most central part of a view (...in hand-held frame)
b. Information closing to the boundary is not important (... in Kinect frame)
3. Re-Confirming the metric for similarity computation:
- Use SAD + NearestNeighborRatio
- SURF features in 128-length extracting from each MSER feature location
4. Part of results in scaling cases (...basic cases): number of matches are around 20 to 90
5. Part of results in negative cases: number of matches are lower than 10
II. Part of results in changing viewpoint:
- number of matches has the similar trend as shown on the top 2 cases
III. Next... Considering to introduce Color and Depth information from the RGB-D camera, i.e. Kinect camera, seeing whether make something different in distinguishing position and negative cases
1. Deal with the 'n-to-1' cases: keep the most important one
2. Make an attention area in both Kinect frame and hand-held camera frame:
- keep the information which is 60% of the original frame (...just use the central 70% information)
- Reason:
a. A person's visual attention should be at the most central part of a view (...in hand-held frame)
b. Information closing to the boundary is not important (... in Kinect frame)
3. Re-Confirming the metric for similarity computation:
- Use SAD + NearestNeighborRatio
- SURF features in 128-length extracting from each MSER feature location
4. Part of results in scaling cases (...basic cases): number of matches are around 20 to 90
| Scaling 1: before improve |
| Scaling 1: after improve |
| Scaling 2: before improve |
| Scaling 2: after improve |
![]() |
| Scaling 5: before improve |
![]() |
| Scaling 5: after improve |
5. Part of results in negative cases: number of matches are lower than 10
II. Part of results in changing viewpoint:
- number of matches has the similar trend as shown on the top 2 cases
III. Next... Considering to introduce Color and Depth information from the RGB-D camera, i.e. Kinect camera, seeing whether make something different in distinguishing position and negative cases
Friday, 25 April 2014
Work of Camera Calibration (13)
I. Run part of our datasets in order to confirm which setup is going to be taken for matching:
- The final decision is:
1. Use MSER descriptor to locate the possible location of useful feature:
- a region-based method
- set 'RegionAreaRange' to [5 RAR*0.3] in order to maintain global information
2. Use SURF descriptor as a description for each feature generating from last step:
- a blob-based method
- using 128 as the length of a descriptor, which considers its orientation comparing to '64'
- Explanation of SURF in OpenCV
3. Use 'SAD' and 'NearestNeighborSymmetric' metric to compute to confirm similarities among
Kinect frames and Hand-held camera frames.
4. This is how the descriptor looks like:
- Part of Results in doing matching under different scaling after improvement and modification the
computation metrics as well as parameters (Right: Kinect frame; Left: Hand-held camera)
II. I would like to use some values such as 'SignOfLaplacian' and 'Orientation' in my modified
matching process.
- Current process in Matching is able to make the correspondence becoming much more precise and concentrate. Yet it is for describe how similar between a Kinect grames and a Hand-held camera frame. I need additional metrics to describe how different between these frames. (Right: Kinect frame; Left: Hand-held camera)
- Eliminate 1-to-multiple relationship in the matching results... may need to sort out
- Check whether the following information is useful or not:
- The final decision is:
1. Use MSER descriptor to locate the possible location of useful feature:
- a region-based method
- set 'RegionAreaRange' to [5 RAR*0.3] in order to maintain global information
2. Use SURF descriptor as a description for each feature generating from last step:
- a blob-based method
- using 128 as the length of a descriptor, which considers its orientation comparing to '64'
- Explanation of SURF in OpenCV
3. Use 'SAD' and 'NearestNeighborSymmetric' metric to compute to confirm similarities among
Kinect frames and Hand-held camera frames.
4. This is how the descriptor looks like:
- Part of Results in doing matching under different scaling after improvement and modification the
computation metrics as well as parameters (Right: Kinect frame; Left: Hand-held camera)
II. I would like to use some values such as 'SignOfLaplacian' and 'Orientation' in my modified
matching process.
- Current process in Matching is able to make the correspondence becoming much more precise and concentrate. Yet it is for describe how similar between a Kinect grames and a Hand-held camera frame. I need additional metrics to describe how different between these frames. (Right: Kinect frame; Left: Hand-held camera)
- Eliminate 1-to-multiple relationship in the matching results... may need to sort out
- Check whether the following information is useful or not:
III. Adaptive method to decide the threshold for filtering out inappropriate matches instead of
using a fix value
Subscribe to:
Posts (Atom)
































