Optical hand gesture recognition sees improvements in accuracy and complexity thanks to new algorithm

In the hit 2002 science fiction film Minority report, Tom Cruise’s character John Anderton uses his hands, sheathed in special gloves, to interface with his transparent, wall-sized computer screen. The computer recognizes his gestures to enlarge, zoom and swipe. Although this futuristic vision of human-computer interaction is now 20 years old, today’s humans still interface with computers using a mouse, keyboard, remote control or a small touch screen. However, many efforts have been devoted by researchers to unlock more natural forms of communication without requiring contact between the user and the device. Voice commands are a striking example that have found their way into modern smartphones and virtual assistants, allowing us to interact and control devices through speech.

Hand gestures are another important mode of human communication that could be adopted for human-computer interactions. Recent advances in camera systems, image analysis and machine learning have made optical gesture recognition a more attractive option in most settings than approaches relying on wearable sensors or data gloves. , as used by Anderton in Minority report. However, current methods are hampered by various limitations, including high computational complexity, low speed, poor accuracy, or low number of recognizable gestures. To solve these problems, a team led by Zhiyi Yu from Sun Yat-sen University, China, recently developed a new hand gesture recognition algorithm that strikes a good balance between complexity, accuracy and applicability. As detailed in their article, which was published in the Electronic Imaging Journal, the team adopted innovative strategies to overcome key challenges and realize an algorithm that can be easily applied to consumer devices.

One of the main characteristics of the algorithm is the adaptability to different types of hands. The algorithm first tries to classify the user’s hand type as thin, normal, or wide based on three measurements considering the relationships between palm width, palm length, and finger length . If this classification is successful, subsequent steps in the hand gesture recognition process compare only the input gesture with stored samples of the same hand type. “Traditional simple algorithms tend to suffer from low recognition rates because they cannot handle different hand types. By first classifying the input gesture by hand type and then using sample libraries to match this type, we can improve the overall recognition rate with almost negligible resource consumption,” says Yu.

Another key aspect of the team’s method is the use of a “shortcut function” to perform a prerecognition step. While the recognition algorithm is able to identify an input gesture among nine possible gestures, comparing all the features of the input gesture with those of the stored samples for all possible gestures would be very time consuming. To solve this problem, the pre-recognition step calculates a ratio of the surface of the hand to select the three most probable gestures among the nine possible ones. This simple feature is enough to reduce the number of candidate gestures to three, among which the final gesture is decided using a much more complex and high-precision feature extraction based on “Hu invariant moments”. Yu says, “The gesture pre-recognition step not only reduces the amount of computation and hardware resources required, but also improves recognition speed without compromising accuracy.

The team tested their algorithm on both a commercial PC processor and an FPGA platform using a USB camera. They asked 40 volunteers to perform the nine hand gestures multiple times to build the sample library, and another 40 volunteers to determine the accuracy of the system. Overall, the results showed that the proposed approach could recognize hand gestures in real time with an accuracy greater than 93%, even if the input gesture images were rotated, translated or scaled. . According to the researchers, future work will focus on improving the performance of the algorithm in poor lighting conditions and increasing the number of possible gestures.

Gesture recognition has many promising application areas and could pave the way for new ways to control electronic devices. A revolution in human-computer interaction may be at hand!

Source of the story:

Material provided by SPIE – International Society for Optics and Photonics. Note: Content may be edited for style and length.