Industrial application of machine-in-the-loop-learning for a medical robot vision system - Concept and comprehensive field study

The availability of increasingly powerful sensors and processing hardware provides the means for performing more and more complex signal processing and recognition tasks. In particular, vision systems have been the subject of intensive research and successful application during the last two decades, whereby they have often been augmented by auxiliary sensors. However, several fundamental challenges concerning the design and application of such systems remain. From the plethora of off-the-shelf-hardware, sensors and algorithms which is available a selection must be made which results in a feasible and well performing system which at the same time is capable of meeting constraints such as cost and power consumption, for instance. Currently, the design process is undergoing a paradigm shift from expert driven to automated design. Learning and optimisation techniques have proved to be more than able to compete with expert based solutions in many cases. As soon as a developed solution is deployed to more than one installation site and operated for an extended period of time under significantly varying environmental conditions, the initial solution will no longer be applicable and adjustment or fine-tuning will be required. This paper presents a conceptual solution to this challenge for the application of a medical robot vision system for the task of tube type detection. Laboratories employ this robot system for semi-automated or automated analysis of medical samples at numerous sites. The varying environmental conditions and variations in the classification task from situation to situation require reconfiguration capabilities. In our approach, these capabilities are realised in the form of a holistic training concept, designated as machine-in-the-loop-learning (MILL). The current state of concept implementation in the robot vision system with MILL support has allowed for its successful implementation in more than 250 machines at 150 sites worldwide with more than 5000 classifications per day and machine. An error rate of 0.06% was reported in a long-term study of three months duration involving two representative installations. Future work will focus on the refinement of the system architecture and incorporation of recent advances in optimisation, e.g., particle swarm optimisation (PSO) and related methods.