Calibration-free augmented reality in perspective

This paper deals with video-based augmented reality and proposes an algorithm for augmenting a real video sequence with views of graphics objects without metric calibration of the video camera by representing the motion of the video camera in projective space. A virtual camera, by which views of graphics objects are generated, is attached to a real camera by specifying image locations of the world coordinate system of the virtual world. The virtual camera is decomposed into calibration and motion components in order to make full use of graphics tools. The projective motion of the real camera recovered from image matches has the function of transferring the virtual camera and makes the virtual camera move according to the motion of the real camera. The virtual camera also follows the change of the internal parameters of the real camera. This paper shows the theoretical and experimental results of our application of nonmetric vision to augmented reality     

Visual creation of inhabited 3D environments

The creation of virtual reality applications and 3D environments is a complex task that requires good programming skills and expertise in computer graphics and many other disciplines. The complexity increases when we want to include complex entities such as virtual characters and animate them. In this paper we present a system that assists in the tasks of setting up a 3D scene and configuring several parameters affecting the behavior of virtual entities like objects and autonomous virtual humans. Our application is based on a visual programming paradigm, supported by a semantic representation, an ontology for virtual environments. The ontology allows us to store and organize the components of a 3D scene, together with the knowledge associated with them. It is also used to expose functionalities in the given 3D engine. Based on a formal representation of its components, the proposed architecture provides a scalable VR system. Using this system, non-experts can set up interactive scenarios with minimum effort; no programming skills or advanced knowledge is required.     

A transatlantic research and development environment [3D virtualgraphics]

Distributed 3D virtual environments can help researchers conduct experiments globally with remotely located participants. We discuss challenges and opportunities for the shared work environment. Our world is entering an age where our current understanding of telecommunications and graphics computing is constantly challenged. With the availability of global information highways, 3D graphical intercontinental collaboration will become a part of our daily work routine. Already, world class auto makers are attempting to reduce car development time to two-year cycles, enlisting global engineering teams. However, this process requires new tools such as shared 3D CAD and distributed product data management systems. The Fraunhofer Center for Research in Computer Graphics (CRCG) in the United States and the Fraunhofer Institute for Computer Graphics (IGD) in Germany, are looking ahead into this new age by establishing a transcontinental computer graphics research effort and a proposed G-7 testbed. We are studying how 3D computer graphics and virtual environments can aid global collaborative work. We have focused our research efforts on determining how computer networks can transform the distributed workplace into a shared environment, allowing real time interaction among people and processes without regard to their location     

Dynamic Comics for Hierarchical Abstraction of 3D Animation Data

Image storyboards of films and videos are useful for quick browsing and automatic video processing. A common approach for producing image storyboards is to display a set of selected key‐frames in temporal order, which has been widely used for 2D video data. However, such an approach cannot be applied for 3D animation data because different information is revealed by changing parameters such as the viewing angle and the duration of the animation. Also, the interests of the viewer may be different from person to person. As a result, it is difficult to draw a single image that perfectly abstracts the entire 3D animation data. In this paper, we propose a system that allows users to interactively browse an animation and produce a comic sequence out of it. Each snapshot in the comic optimally visualizes a duration of the original animation, taking into account the geometry and motion of the characters and objects in the scene. This is achieved by a novel algorithm that automatically produces a hierarchy of snapshots from the input animation. Our user interface allows users to arrange the snapshots according to the complexity of the movements by the characters and objects, the duration of the animation and the page area to visualize the comic sequence. Our system is useful for quickly browsing through a large amount of animation data and semi‐automatically synthesizing a storyboard from a long sequence of animation.     

Event,State, And Process In Arrow Logic

Artificial agents, which are embedded in a virtual world, need to interpret a sequence of commands given to them adequately, considering the temporal structure for each command. In this paper, we start with the semantics of natural language and classify the temporal structures of various eventualities into such aspectual classes as action, process, and event. In order to formalize these temporal structures, we adopt Arrow Logic. This logic specifies the domain for the valuation of a sentence as an arrow. We can connect, or give order to, arrows by defining inter-arrow operations, and can give different views for sentences. Thereafter we formalize the rules of aspectual shifts in situated inference, in the style of a logic programming language. Thus, we not only describe the static representation of temporal features, but also show the dynamic process to deduce how each eventuality is viewed. The rules are applied to the information flow through the sequence of commands; therefore, we consider how the temporal structure of a command affects the succeeding commands.     

An overview of physically-based modelling techniques for virtual environments

A class of virtual environments is concerned with the representation of behaviour that is apparent in the real world. In order to model this behaviour, sophisticated physical models are required. The development of these models, classed asphysically-based modelling, is based upon the fundamental concepts of Newtonian dynamics. Considerable research into physically-based modelling has already been conducted by the computer graphics community, permitting realistic animation of object motion. The application of physical models to virtual environments poses further problems, not least that of real-time execution in a fully interactive environment. This paper gives an overview of the existing computer graphics research concerned with physically-based modelling, discussing the merits and problems of various techniques in terms of the requirements of virtual environment.     

Automated camera planning to film robot operations

Automatic 3D animation generation techniques are becoming increasingly popular in different areas related to computer graphics such as video games and animated movies. They help automate the filmmaking process even by non professionals without or with minimal intervention of animators and computer graphics programmers. Based on specified cinematographic principles and filming rules, they plan the sequence of virtual cameras that the best render a 3D scene. In this paper, we present an approach for automatic movie generation using linear temporal logic to express these filming and cinematography rules. We consider the filming of a 3D scene as a sequence of shots satisfying given filming rules, conveying constraints on the desirable configuration (position, orientation, and zoom) of virtual cameras. The selection of camera configurations at different points in time is understood as a camera plan, which is computed using a temporal-logic based planning system (TLPlan) to obtain a 3D movie. The camera planner is used within an automated planning application for generating 3D tasks demonstrations involving a teleoperated robot arm on the the International Space Station (ISS). A typical task demonstration involves moving the robot arm from one configuration to another. The main challenge is to automatically plan the configurations of virtual cameras to film the arm in a manner that conveys the best awareness of the robot trajectory to the user. The robot trajectory is generated using a path-planner. The camera planner is then invoked to find a sequence of configurations of virtual cameras to film the trajectory.     

Succinct Representation of Labeled Graphs

In many applications, the properties of an object being modeled are stored as labels on vertices or edges of a graph. In this paper, we consider succinct representation of labeled graphs. Our main results are the succinct representations of labeled and multi-labeled graphs (we consider planar triangulations, planar graphs and k-page graphs) to support various label queries efficiently. The additional space cost to store the labels is essentially the information-theoretic minimum. As far as we know, our representations are the first succinct representations of labeled graphs. We also have two preliminary results to achieve the main contribution. First, we design a succinct representation of unlabeled planar triangulations to support the rank/select of edges in ccw (counter clockwise) order in addition to the other operations supported in previous work. Second, we design a succinct representation for a k-page graph when k is large to support various navigational operations more efficiently. In particular, we can test the adjacency of two vertices in O(lg?k) time, while previous work uses O(k) time.     

基于真实环境图象的三维动态仿真系统研究

针对在城市规划中,建筑设计方案的计算机三维图形与真实环境图象进行匹配及合成的问题,研究和提出了一种基于摄影测量学返透视原理的,对实际环境摄像经抽样采集的图象序列进行亮度、方差及平均色度合成的图象分析方法,并研究了一种基于单象空间后方交会解析摄影测量技术一阶渐近迭代计算的求解虚拟摄像参数的算法,同时提出了基于关键帧技术的适合于图象合成时,计算分割区域随时间变化的动态掩模算法,还在太阳系行星运动规律的基础上,建立了适合于计算机虚拟三维世界坐标系的日照模型,以及相应的求解任意时刻、任意经纬度虚拟世界坐标位置处的虚拟太阳光源方向和强度的算法。这些算法已被应用到基于真实环境图象的三维动态仿真系统中,实验结果表明,这些算法是行这有效的。     

3D technology, ready for the PC?

3D graphics software is computationally intensive, but the hardware is getting cheaper. Suitable motherboard-bus and graphics card developments are discussed, as are ways of overcoming network-bandwidth limitations. The article then considers applications-games and entertainment, design, presentations, and virtual shopping; one can even make a virtual representation of oneself to try on virtual clothes before ordering them. Finally, the development of standards in formats and in players for distribution is discussed     

Depicting dynamics using principles of visual art and narrations

In visual art, a single static image frequently represents much more than projected 3D scenery. Artists include subtle visual elements outlining movements, indicating past or future events, sketching ongoing activities, or guiding the observer's attention. Artists have found ways to visualize physical as well as nonphysical dynamics of scenes using graphics techniques. In a sense, we can consider these smart depictions - a form of expressive, visual content adopting the styles of visual art and abstraction techniques. These depictions can serve, for example, as pictograms and signs that advise and assist people. They're also present in comic books and storyboards that effectively present dynamics and narrate sequential processes. Taking smart depictions a step further, we created a system that automatically generates smart, compelling images of 3D scenes that depict dynamics following traditional design principles of visual art, visual narrations, and graphic design, such as those found in comic books and storyboards. These media offer a rich vocabulary of visual art deployed as techniques to facilitate visual communication of a wealth of activities and events in static images. In particular, we can symbolize in a single, static image past, ongoing, and future activities as well as events taking place in 3D scenes. Additionally, we can take into account nonvisual information.     

Complex RFID event processing

Advances of sensor and radio frequency identification (RFID) technology provide significant new power for humans to sense, understand and manage the world. RFID provides fast data collection with precise identification of objects with unique IDs without line of sight, thus it can be used for identifying, locating, tracking and monitoring physical objects. Despite these benefits, RFID poses many challenges for data processing and management: (i) RFID observations have implicit meanings, which have to be transformed and aggregated into semantic data represented in their data models; and (ii) RFID data are temporal, streaming, and in high volume, and have to be processed on the fly. Thus, a general RFID data processing framework is needed to automate the transformation of physical RFID observations into the virtual counterparts in the virtual world linked to business applications. In this paper, we take an event-oriented approach to process RFID data, by devising RFID application logic into complex events. We then formalize the specification and semantics of RFID events and rules. We discover that RFID events are highly temporal constrained, and include non-spontaneous events, and develop an RFID event detection engine that can effectively process complex RFID events. The declarative event-based approach greatly simplifies the work of RFID data processing, and can significantly reduce the cost of RFID data integration. This work was done by F. Wang while working at Siemens Corporate Research. This work was done by S. Liu while visiting Siemens Corporate Research.     

交互式虚拟内窥镜系统

计算机图形图象技术与虚拟现实技术应用于医学内窥镜系统,产生了虚拟内窥镜技术,为了将虚拟现实技术应用在医学图象处理方面,以方便医生进行虚拟手术与无创诊断,在综合利用各种计算机图形,图象技术的基础上,提出了完整的交互式虚拟内窥镜系统的框架,同时对系统结构和各种模型进行了分析和讨论,还针对系统的实时性和绘制结果的逼真性要求,提出了基于Object Cache和扩展的区域增长方法,并将其应用到医学图象处理当中,得到了较好的效果,该系统较好地解决了虚拟现实与可视化实时性和绘制精度两方面的要求,从而为医学图象可视化提供了有力的工具。     

High‐resolution 360 Video Foveated Stitching for Real‐time VR

In virtual reality (VR) applications, the contents are usually generated by creating a 360° Video panorama of a real‐world scene. Although many capture devices are being released, getting high‐resolution panoramas and displaying a virtual world in real‐time remains challenging due to its computationally demanding nature. In this paper, we propose a real‐time 360° Video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high‐resolution panoramic Video in real‐time that can be streamed directly to the client. Our foveated stitching algorithm takes Videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.     

Preface

This paper discusses the problems in teleoperation systems for a mobile robot and the utilization of a virtual world in such systems. In order to achieve smooth operation of the mobile robot through a communication link, we should consider time delays in data transfer. To compensate for the incomplete data sets, the virtual images can be generated by computer graphics when the information on the working environment can be acquired beforehand. In this paper, we construct a teleoperation system with a virtual world. The performance of the system is examined through experiments with actual mobile robots which show that the virtual robot can be operated by an operator in almost the same manner as the teleoperated real robot. In an experimental environment with a second moving robot, we can keep the status of the second robot under perfect control and operate the first robot with no interference.     

Information at a glance [augmented reality user interfaces]

What if we could visualize and interact with information directly in the context of our surroundings? Our research group is exploring how augmented reality (AR) could someday make this possible. AR integrates a complementary virtual world with the physical world-for example, by using head-tracked see-through head-worn displays to overlay graphics on what we see. Instead of looking back and forth between the real world and a PDA, we look directly at the real world and the virtual information overlaid on it. At the heart of this approach is context-aware computing, computing systems that are sensitive to the context in which they operate, ranging from human relationships to physical location. For example, information might be tied to specific locations within a global, Earth-centered, coordinate system. How can we design effective mobile AR user interfaces? We've been trying to answer this question in part by developing experimental AR research prototypes. In AR, as in work on information visualization using desktop technologies, the amount of information available can far exceed what a system can legibly display at a given time, necessitating information filtering. Julier et al. (2000) have developed information filtering techniques for AR that depend on the user's goals, object importance, and proximity. We assume that a system can accomplish information filtering of this sort and that our system is displaying everything it should.     

Enhanced waypoint graph for surface and volumetric path planning in virtual worlds

Our research focuses on the problem of path planning in 3D virtual world applications. The characters we consider are heterogeneous, as they have different sizes, and can perform surface or volumetric motion. In this paper, we propose an enhanced waypoint graph, which consists of point nodes equipped with radius, as well as edges connecting those nodes. Each edge is labeled with the motion type it can support. Given a polygon soup representation of a virtual world, the proposed algorithm starts by subdividing the virtual world into regions. This enables us to process large virtual worlds. Each region is then locally voxelized, one at a time. Two kinds of waypoints are generated: local waypoints using corner detection on the voxelization, and border waypoints at the region boundary. Waypoints are then sparsely connected to form a local waypoint graph, and local graphs are finally connected via the border waypoints to create the final global enhanced waypoint graph. To plan paths between arbitrary points using this graph, the points are connected to the graph using nearest neighbor search and traversability test, then Dijkstra/A* algorithm is used to calculate the final path, taking into account the appropriate size and motion type.     

Neural network-based detection of virtual environment anomalies

The increasingly widespread use of large-scale 3D virtual environments has translated into an increasing effort required from designers, developers and testers. While considerable research has been conducted into assisting the design of virtual world content and mechanics, to date, only limited contributions have been made regarding the automatic testing of the underpinning graphics software and hardware. In the work presented in this paper, two novel neural network-based approaches are presented to predict the correct visualization of 3D content. Multilayer perceptrons and self-organizing maps are trained to learn the normal geometric and color appearance of objects from validated frames and then used to detect novel or anomalous renderings in new images. Our approach is general, for the appearance of the object is learned rather than explicitly represented. Experiments were conducted on a game engine to determine the applicability and effectiveness of our algorithms. The results show that the neural network technology can be effectively used to address the problem of automatic and reliable visual testing of 3D virtual environments.