Augmented reality-based block piling game with superimposed collapse prediction

Understanding what cannot be seen is difficult. Physical behavior can be explained on the basis of physical theories even if the behavior cannot be observed. Explanation of what is physically happening in the real world would become easy, however, if annotations were superimposed on the real objects. Herein, the authors demonstrate how an understanding of a physical event can be facilitated by overlapping a real-world situation with a simulation that predicts a future state. This idea is demonstrated in a game application in which a player stacks blocks into a pile until it collapses. In general, it is easy to estimate whether a block on the edge of a table will fall or not. However, it is more difficult to predict whether a stack of many blocks will collapse, and in what manner the stack will collapse. Even though previous research has demonstrated that the problem of how two-dimensionally stacked blocks collapse can be reduced to solving a sequence of convex quadratic programs, algorithms for convex quadratic programs require massive computational resources. Hence, the authors developed a fast and new algorithm based on a linear program. The proposed algorithm realizes real-time simulation based on physics that superimposes predicted collapse. The block that is predicted to fall is superimposed on the real block with a lit background projection. The system was evaluated in an experiment, and superimposed augmented reality annotation was observed to be efficient. The system was also demonstrated in game contests and received positive feedback and comments.