Author: Dipl.-Ing. Wanda Benešová, PhD.

Description

Dominant orientation templates (DOT) is a method for real-time object detection, which works well for detection of untextured objects and is related to method Histogram of oriented gradients. DOT is neither based on statistical learning of object’s shapes nor on feature point detection, but it uses real-time Template Matching recognition with locally most dominant orientations from HoG.

OpenCV function used

cvCaptureFromAVI, cvtColor

The process

1. Computation of gradients for each pixel in template and input image
1. Provided by convolution kernel
2. For each pixel
3. Gradient is defined by magnitude and direction
4. 0-180° instead of 0-360° range
5. Directions can be discretized from 0-180° into bins (e.g. 9 bins by 20° )

for (int r=0;r<=area.rows-region_size;r+=region_size)
{
    for (int s=0;s<=area.cols-region_size;s+=region_size)
    {
        int mag=gradienty_template.gradient[i][j].magnitude;
        n0=histogram_group(gradienty_template.gradient[i][j].direction);
        if (mag>min_magnitude)
            template_hist.hist_matrix[hx][hy].bins[n0]+=mag;
    }
}

2. Dividing pixels into regions

   

   for (int r=0;r<=area.rows-region_size;r+=region_size)
       // moving in the picture with step size of 7 or 9
   

3. Computing most dominant gradient orientations for each region

   

   if (template_hist.hist_matrix[i][j].bins[k]>max) //now only the most dominant
   {
       if  (template_hist.hist_matrix[i][j].bins[k]>min_magnitude)
       {
           max=template_hist.hist_matrix[i][j].bins[k];
           max_index=k;
       }
   }
   

4. Template matching and comparing of most dominant orientations.
5. Evaluating comparison.

   

   

Description

The main purpose of this work is to detect eyes and recognize when are open and when close. To execute this purpose we must use video camera or video file with person face.

Eye Detection

To detect eye blinking we need to recognize face and eyes on image. For this intention we use Viola Jones algorithm which detect this features and bounded it with rectangles.

Because this algorithm is performance consuming, we use tracking which is more faster. We use Good Features to Track algorithm which return set of points suitable to track, and then with Lucas-Kanade tracker algorithm we track it on every frame.

// track points
calcOpticalFlowPyrLK(prevGray, gray, features, cornersB, status, error, Size(31, 31), 1000);

if(!calculateIntegralHOG(gray(rectangleFace)))
    text = "CLOSED";

There is problem with points which is not precisely targeted to next frame, so we remove them from set of tracking points. When number of points is not enough to track, we repeat eye detection method again.

Eye Blinking Detection

To detect if eyes are open or close we use HOG descriptor which return array of floats representing lines orientation. Because HOG descriptor is usable only on images with specific resolution, we use sliding window with this resolution which covers our image.

cv::gpu::HOGDescriptor gpu_hog(win_size, Size(16, 16), Size(8, 8), Size(8, 8), 9, 0.8, 0.00015, true);

// calculate HOG for every window
GpuMat gpuMat;
gpuMat.upload(cropped);
GpuMat descriptors;
gpu_hog.getDescriptors(gpuMat, win_size, descriptors);
Mat descriptorMat = Mat(descriptors);

In next step we take array of floats returned from HOG descriptor and transform it to histogram. We notice, that when eye is closed the local maximum of this histogram is much lower than local maximum of opened eye, so we define the value which separate opened and closed eye.

Because we use sliding window, we average all this local maxims and based on returned value, we decide if specified area contains open or close eyes.

Description

The goal of this project is to implement algorithm that finds dots on dices. Motivation was idea / question how to create home-made random number generator? We can throw dices and our application will be able to recognize summary value on dices.

This program uses fitEllipse() function to find dots on dices. The basics steps are as follows:

Process

1. Open video stream

   CvCapture* capture = cvCaptureFromCAM( CV_CAP_ANY );
   

2. For the whole stream we create single frame

   IplImage* frame = cvQueryFrame( capture );
   

3. Invert color
4. Use adaptive threshold

   adaptiveThreshold(image, bimage, 255, ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY, 15, -10);
   

5. We can use morphological operations (dilatation, erosion) to expand/minimize contours
6. To find circles we use following:

   Mat pointsf;
   Mat(contours[i]).convertTo(pointsf, CV_32F);
   RotatedRect box = fitEllipse(pointsf);
   

7. If difference between box.size.width and box.size.height is lower than treshold, we consider ellipse as circle.
8. At this point we have a lot of “circles”. An experimenting helped us to determine, which circle in the picture is real point on dice. Based on size of real dices points we can isolate only real dices points.

Example of process

Using custom settings we are able to improve results in specific situations.

findContours(bimage, contours, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);

Vector<pair<RotatedRect*, int>> vec, finalVec;
Mat cimage = Mat::zeros(bimage.size(), CV_8UC3);
int i;
int w, h, wAhThr, angleThr, centerThr;
int  hwDifferenceThreshold, histThreshold;						
centerThr = settCenterThr;
wAhThr = settwAhThr;
hwDifferenceThreshold = settHWdifferenceThr;
histThreshold = settHistogramThr;

for(i = 0; i < contours.size(); i++)
{
	size_t count = contours[i].size();
	if( count > 50 || count  < 6)
	      continue;

	Mat pointsf;
	Mat(contours[i]).convertTo(pointsf, CV_32F);
	RotatedRect box = fitEllipse(pointsf);

	w = box.size.width;
	h = box.size.height;

	int hwDifference = abs(h - w);
	if (hwDifference > hwDifferenceThreshold)
	      continue;

	if (w < wAhThr || h < wAhThr)
	      continue;

	vec.push_back(pair<RotatedRect*, int>(new RotatedRect(box), i));
}

int asdf = vec.size();
Vector<pair<RotatedRect* ,int>>::iterator it ,iend, it2;
int MAXHIST = 200;
int* histVals = new int[MAXHIST];
for (int i = 0; i < MAXHIST; i++)
	histVals[i] = 0;

int histIter = 0;
RotatedRect * box;
RotatedRect * box2;
int maxWidth = 0;
int distanceOfCenters;
for (it = vec.begin(), iend = vec.end(); it != iend; it++)
{
	box = (it->first);
	for (it2 = it + 1; it2 != iend; it2++)
	{
		box2 = (it2->first);
		distanceOfCenters = (int)std::sqrt((box->center.x - box2->center.x) * (box->center.x - box2->center.x) + (box->center.y - box2->center.y)  * (box->center.y - box2->center.y));
		if (distanceOfCenters < centerThr)
		{
			if (box->size.width > box2->size.width)
				it2->second = -1;
			else
				it->second = -1;
			break;
		}

	}
}