Hand Video Detection

In this proposed method, firstly RGB images are captured by the camera, but the RGB color space is not adequate to compare colors, so the color space is changed to HSV. The HSV representation models the way paints of different colors mix together, with the saturation dimension resembling various shades of brightly colored paint, and the value dimension resembling the mixture of those paints with varying amounts of black or white paint.[1] Before finding the bounding box we have applied some of noise elimination steps such as threshold, dilation, erosion, and blur.

1. In threshold, we threshold the image with the skin color values to only get the defined color of the hand;
2. In dilation, we extrapolate the hand to fill dark spots within by increasing the white area;
3. In erosion, because the object was expanded in the dilation phase, we need to shrink it to have a better notion of the contour of the hand;
4. In the blur, by convolving the image with a low-pass filter kernel, we remove noises. We use Gaussian blur, as it is highly effective in removing Gaussian noise and is not heavy, but the edges are blurred a little bit in this operation[2].

After processing the image, the contours are calculated. Contours can be explained simply as a curve joining all the continuous points, along with the boundary, having the same color or intensity. The contour with the biggest area should be the hand to be detected. Then we do an approximation of the contours by using the Ramer-Douglas-Peucker algorithm. That iterative end-point fit algorithm is an algorithm that decimates a curve composed of line segments to a similar curve with fewer points.[3]

Then we make a convex hull around the hand to detect any convexity defects, that means, all the areas that do not belong to the object but are located inside of its convex hull. The convexity defects, in this case, are recognized as the space between fingers. The space between fingers is inside the convexity hull but doesn’t match the hand color.

Then, by analyzing the number of defects, the area of the convexity defects, the area of the hull, and the ratio between the area of hand and hull we can determinate what signal is the hand doing.