Assessment Of Gesture-based Natural Interface Systems In Serious Games

The goal of this study was to examine interaction designs and devices for natural gesture-based interaction for a 3D serious game for surgical training. The 3D input devices used in the experiment consisted of a 5DT Data Glove and a Leap Motion sensor. The 3D model used for interaction was a particle-based deformable object using NVIDIA Flex. The interface was constructed in C++ using Unreal Engine 4 as the game engine and was designed to measure the movement of the wrist around a virtual object. A user study was conducted to compare the usability of contact based (5DT Glove) and non-contact based (the Leap) gesture-based input devices for use in the system. Eleven right-handed adult volunteers were recruited. Each volunteer was asked to perform four tasks on the deformable model. These tasks involved pushing and pulling certain points and faces on the 3D model. To account for possible learning bias, subjects were randomly assigned their first device. After each device, the subjects were asked to complete a System Usability Scale questionnaire. The experiment collected information on the (a) accuracy, (b) discoverability, and (c) ease of use of the two devices. Accuracy was measured based on the distance between the user’s selected location and a predetermined target point. Discoverability was based on the time taken to reach the selection point. Ease of use was determined by survey. Standard descriptive statistics are employed (mainly means and standard deviations). The results indicate that while non-contact based devices might be perceived as having greater ease of use due to the lack of restrictions and cabling when compared to the glove, the glove was more accurate. The favourability of the glove among those who preferred the glove was higher than the favourability of the Leap among those who preferred the Leap. The significance of this is dependant upon the type of learning scenario employed in the serious game. If the emphasis within the serious game is on task training, e.g. dexterous tasks associated with surgical instrument manipulation, then the results suggest the use of the Glove is more appropriate due its higher accuracy count. However, if the focus in the serious game is on naturalness of interaction then the Leap is more highly preferred. These insights may help serious game designers and educational content creators in adopting the appropriate input device for natural gesture-based interaction.