A multimodal teaching advisor for sensor-enhanced robotic systems in manufacturing

A multimodal teaching advisor (MTA) has been developed for sensor-enhanced robotic systems used in manufacturing. The MTA utilizes the work-site operator's know-how and robotic-systems information, including that from sensors, in a complementary manner. This system integrates information from various sources to carry out robotic tasks and presents this information through user interfaces that are easily understood by the operator. That is, the system integrates task specifications including tolerances, system constraints, sensory information, and operator- and robot-related information. The synthesized information is then presented in realtime to the operator at the work-site through a mobile multimodal interface. Experimental results show that this system can significantly improve total robotic system performance by ensuring a high-quality teaching task.