Michael Wiznitzer

Technical Skills Used

• Python
• Image processing & Object Detection using the OpenCV library
• Histogram-based Skin Color Segmentation from Training Images
• Gesture Detection
• Morphological Operations (Dilation, Erosion)
• Gaussian Blurring
• Game Interfacing using the Pygame library

How it Works

To understand the rules of the game and how it's played, check out my team's Readme.

As is mentioned in the Readme, skin tone training data is captured from the player during the calibraiton phase of the game. The image is transformed into the HSV color space so that the HS histograms could be determined. These histograms provide the range of H and S values that should be expected from the player's hand during game play. Consequently, this makes the game robust and faster since it automatically takes into account the ambient lighting and the skin tone specific to the player's hand.

In order to reduce noise on the input images, the following morphological operation sequence is done:

• Gaussian Blurring to reduce the weight of outlier pixels
• Erosion to get rid of any noise left behind by the blurring process
• Dilation to fill in any gaps that should now mostly exist on the target image and not around noisy pixels (which should be at a minimum after the previous 2 processes)

Gesture recognition was acheived by using the ConvexHull function from OpenCV. This provides the points that make up the polygon that surrounds a player's hand. The first and second distinct maximum distances from the hand centroid (which is denoted by the blue dot) represent the location of the index and thumb fingertips respectively. Then by knowing the typical ratio of the distance-to-the-thumb-tip over the distance-to-the-index-fingertip for both the open-hand and shooting-hand configurations, hand gestures could be determined.

Demo