Object recognition (RANSAC verification)

This project shows object recognition using local features-based methods. We use four methods for keypoints detection and description: SIFT/SIFT, SURF/SURF, FAST/FREAK and ORB/ORB. Keypoints are used to compute homography. Object is located in scene with RANSAC algorithm. RGB and hue-saturation histograms are used for RANSAC verification.

Functions used: FeatureDetector::detect, DescriptorExtractor::compute, knnMatch, findHomography, warp, calcHist, compareHist

Input

The process

1. Keypoints detection

   FeatureDetector * detector;
   detector = new SiftFeatureDetector();
   detector->detect( image, key_points_image );
   
   DescriptorExtractor * extractor;
   extractor = new SiftDescriptorExtractor();
   extractor->compute( image, key_points_image, des_image );
   

2. Keypoints description
3. Keypoints matching

   DescriptorMatcher * matcher;
   matcher = new BruteForceMatcher<L2<float>>();
   matcher->knnMatch(des_object, des_image, matches, 2);
   

4. Calculating homography

   findHomography( obj, scene, CV_RANSAC );
   

5. Histograms matching

   calcHist( &hsv_img_object, 1, channels, Mat(), hist_img_object, 2, histSize, ranges, true, false );
   compareHist( b_hist_object, b_hist_quad, CV_COMP_BHATTACHARYYA );
   

6. Outline recognized object

Sample

Detecting keypoints

Finding matches

Object recognition and RANSAC verification (green outline)

Object recognition and RANSAC failure (red outline)

drawMatches( gray_object, key_points_object, image,
             key_points_image, good_matches, img_matches,
             Scalar::all(-1), Scalar::all(-1), vector<char>(),
             DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );

	if (good_matches.size() >= 4)
	{
	for( int i = 0; i < good_matches.size(); i++ )
	{
	//
	obj.push_back( key_points_object[ good_matches[i].queryIdx ].pt );
	scene.push_back( key_points_image[ good_matches[i].trainIdx ].pt );
	}

	H = findHomography( obj, scene, CV_RANSAC );

	perspectiveTransform( obj_corners, scene_corners, H);
	//*******************************************************

	Mat quad = Mat::zeros(rgb_object.rows, rgb_object.cols,
                   CV_8UC3);

	//warping object back to tamplate rotation
	warpPerspective(frame, quad, H.inv(), quad.size());

	...