Eye Blinking Detection

Description

The main purpose of this work is to detect eyes and recognize when are open and when close. To execute this purpose we must use video camera or video file with person face.

Eye Detection

To detect eye blinking we need to recognize face and eyes on image. For this intention we use Viola Jones algorithm which detect this features and bounded it with rectangles.

Because this algorithm is performance consuming, we use tracking which is more faster. We use Good Features to Track algorithm which return set of points suitable to track, and then with Lucas-Kanade tracker algorithm we track it on every frame.

// track points
calcOpticalFlowPyrLK(prevGray, gray, features, cornersB, status, error, Size(31, 31), 1000);

if(!calculateIntegralHOG(gray(rectangleFace)))
    text = "CLOSED";

There is problem with points which is not precisely targeted to next frame, so we remove them from set of tracking points. When number of points is not enough to track, we repeat eye detection method again.

Eye Blinking Detection

To detect if eyes are open or close we use HOG descriptor which return array of floats representing lines orientation. Because HOG descriptor is usable only on images with specific resolution, we use sliding window with this resolution which covers our image.

cv::gpu::HOGDescriptor gpu_hog(win_size, Size(16, 16), Size(8, 8), Size(8, 8), 9, 0.8, 0.00015, true);

// calculate HOG for every window
GpuMat gpuMat;
gpuMat.upload(cropped);
GpuMat descriptors;
gpu_hog.getDescriptors(gpuMat, win_size, descriptors);
Mat descriptorMat = Mat(descriptors);

In next step we take array of floats returned from HOG descriptor and transform it to histogram. We notice, that when eye is closed the local maximum of this histogram is much lower than local maximum of opened eye, so we define the value which separate opened and closed eye.

Because we use sliding window, we average all this local maxims and based on returned value, we decide if specified area contains open or close eyes.