Alpha-numeric hand gesture recognition based on fusion of spatial feature modelling and temporal feature modelling

Abstract

Alpha-numeric gesture refers to writing in air of alphabet and numeric characters. With prevalent usage of vision enabled smart devices, these gestures are considered as an alternative user interface. As each individual has a unique handwriting style, it has been observed that alpha-numeric gesturing also exhibits different individualistic styles, posing a challenge to the vision based gesture recognition. In this Letter, a simple but effective method of modelling alpha-numeric hand gestures by fusing temporal-feature-state modelling and total-trajectory-shape modelling is proposed. The proposed method employs a convolution neural network that represents total-trajectory-shapes, and combines it with conventional conditional random fields based temporal-feature-state modelling. The proposed algorithm is evaluated in public database of both alphabet and numeric hand gestures. Experimental results show a performance improvement of the proposed algorithm compared with the state-of-the art methods.