Sensory-motor enhancement in a virtual therapeutic environment

The sensory-motor skills of persons with neuromuscular disabilities have been shown to be enhanced by intensive and repetitive therapeutic interventions. This paper describes a form of low immersion virtual reality and a prototype, open source system that allow a user with significant physical disability to actively interact with computer-generated objects whose behaviors promote a game-like interaction. Unlike fully immersive and haptic virtual reality, this approach frees the user from head-mounted displays and gloves. It extracts the user’s real-time silhouette from the output of a remote video camera and uses that two-dimensional outline to interact with graphical objects on screen. In contrast to video games that have been modified with specialized interfaces, this virtual interaction system promotes the repetitive use of goal directed movements of the arms and body, which are essential to promote cortical reorganization, as well as discourage unwanted changes in muscle tissue that result in contracture. A prototype system demonstrates the potential of low immersion technology to motivate users and encourage participation in therapy. It also offers the potential of accommodating the sensory-motor skills of individuals with very significant impairment. The behaviors of the computer-generated graphics can be altered to allow use by those with very limited range of motion and/or motor control. These behaviors can be adjusted to provide a continuing challenge as the user’s skills improve. This prototype is described in terms of functional capabilities that include a silhouette extraction from a video image, and generation of graphical objects that interact with the silhouette. The work is extended with a discussion of a more sophisticated region of interest detection algorithm that can select specific parts of the body.