Extracting the position of game board & recognition of game board pieces

This project focuses on the usage of computer vision within the field of board games. We propose a new approach for extracting the position of game board, which consists of the detection of empty fields based on the contour analysis and elipse fitting, locating the key points by using probabilistic Hough lines and of finding the homography by using these key points.

Functions used: Canny, findContours, fitEllipse, HoughLinesP, findHomography, warpPerspective, chamerMatching

Input

The process

1. Canny edge detector

   Mat canny;
   Canny(img, canny, 100, 170, 3);
   

2. Contour analysis – extraction contours and filtering out those that don’t match our criteria

   vector<vector<Point>> contours;
   vector<Vec4i> hierarchy;
   findContours(canny, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_NONE);
   

3. Ellipse fitting – further analysis of contours, final extraction of empty fields

   RotatedRect e = fitEllipse(contours[i]);
   

4. Extraction of the game board model – 4 key points are needed for locating this model
5. Locating the key points in the input image – using Hough lines & analysing their intersections

   Mat grayCpy;
   vector<Vec4i>& lines;
   HoughLinesP(grayCpy, lines, 1, CV_PI/180, 26, 200, 300);
   

6. Finding the homography and final projection of the game board model into the input image

   findHomography(Mat(modelKeyPoints), Mat(keyPoints));
   warpPerspective(modelImg, newImg, h, Size(imgWithEmptyFieldsDots.cols, imgWithEmptyFieldsDots.rows), CV_INTER_LINEAR + CV_WARP_FILL_OUTLIERS);
   chamerMatching(canny, piece, results, costs, 1.0, 30, 1.0, 3, 3, 5, 0.9, 1.1);
   

Sample

Result

Mat findImageContours(const Mat& img, vector<vector<Point> >& contours, vector<Vec4i>& hierarchy)
{
    // detect edges using canny:
    Mat canny;
    Canny(img, canny, 100, 170, 3);

    findContours(canny, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_NONE);

    // draw contours:   
    Mat imgWithContours = Mat::zeros(canny.size(), CV_8UC3);
    for (unsigned int i = 0; i < contours.size(); i++)
    {
        // process "holes" only:
        if (hierarchy[i][3] == -1) continue;
        // apply ratio + size + contourArea filters:
        if (!checkContour3(contours[i])) continue;

        // fit and draw ellipse:
        RotatedRect e = fitEllipse(contours[i]);
        if (e.size.height < 50)
        {
            line(imgWithContours, e.center, e.center, Scalar(255, 255, 255),3);
        }
    }
    return imgWithContours;
}