Low bandwidth desktop and video streaming for collaborative tiled display environments

High-resolution display environments built on networked, multi-tile displays have emerged as an enabling tool for collaborative, distributed visualization work. They provide a means to present, compare, and correlate data in a broad range of formats and coming from a multitude of different sources. Visualization of these distributed data resources may be achieved from a variety of clustered processing and display resources for local rendering and may be streamed on demand and in real-time from remotely rendered content. The latter is particularly important when multiple users want to concurrently share content from their personal devices to further augment the shared workspace. This paper presents a high-quality video streaming technique allowing remotely generated content to be acquired and streamed to multi-tile display environments from a range of sources and over a heterogeneous wide area network.The presented technique uses video compression to reduce the entropy and therefore required bandwidth of the video stream. Compressed video delivery poses a series of challenges for display on tiled video walls which are addressed in this paper. These include delivery to the display wall from a variety of devices and localities with synchronized playback, seamless mobility as users move and resize the video streams across the tiled display wall, and low latency video encoding, decoding, and display necessary for interactive applications. The presented technique is able to deliver 1080p resolution, multimedia rich content with bandwidth requirements below 10 Mbps and low enough latency for constant interactivity. A case study is provided, comparing uncompressed and compressed streaming techniques, with performance evaluations for bandwidth use, total latency, maximum frame rate, and visual quality.     

Cost-effective haptic-based networked virtual environments with high-resolution tiled display

In this paper, a haptic-based NVE (networked virtual environment) supporting high-resolution tiled display is proposed with a resource management scheme for future home applications such as a haptic-based networked game. Although NVEs with haptic interaction and immersive display have been developed in previous studies, they usually require expensive system configurations. In order to implement a cost-effective system, our proposed design includes a tiled display supporting high-resolution with several low-cost LCDs. The display nodes of the tiled display also have low processing and storage resource requirements because they only need to handle display-ready pixels received from the application node. The proposed resource management scheme, called as an object-based display scheme, reduces the resource requirements of the application node that would otherwise require significant resources to manage all the display nodes and to provide a user with haptic feedback. The proposed scheme segments graphic scenes into several pixelated virtual objects and assigns different frame rates to each object according to the interest of user. Experimental results confirm that the proposed scheme reduces the required network bandwidth and appropriately assigns the limited processing resources to graphic and haptic renderings for a high-level visual and haptic quality.     

Tiled++: An Enhanced Tiled Hi-Res Display Wall

In recent years, high-resolution displays have become increasingly important to decision makers and scientists because large screens combined with a high pixel count facilitate content rich, simultaneous display of computer-generated imagery and high-definition video data from multiple sources. Tiled displays are attractive due to their extended screen real estate, scalability, and low cost. LCD panels are usually preferred over projectors because of their superior resolution. One of the drawbacks of LCD-based tiled displays is the fact that users sometimes get distracted by the screens' bezels, which cause discontinuities in rendered images, animations, or videos. Most conventional solutions either ignore the bezels and display all pixels, causing objects to become distorted, or eliminate the pixels that would normally fall under the bezels, causing pixels to be missing in the display of static images. In animations, the missing pixels will eventually reappear when the object moves, providing an experience that is similar to looking through a French window. In this paper, we present a new scalable approach that leads neither to discontinuities nor to significant loss of information. By projecting onto the bezels, we demonstrate that a combination of LCD-based tiled displays and projection significantly reduces the bezel problem. Our technique eliminates ambiguities that commonly occur on tiled displays in the fields of information visualization, visual data analysis, human-computer interaction, and scientific data display. It improves the usability of multimonitor systems by virtually eliminating the bezels. We describe a setup and provide results from an evaluation experiment conducted on a 3 times 3 and on a 10 times 5 tiled display wall.     

面向多投影显示墙的画面校正技术

多投影显示墙是缓解对超高分辨率显示屏幕不断增长的需求与单台显示设备分辨率有限性之间矛盾的有效手段,但目前还缺乏规范化的模式和方法以支持高精度、高可靠性的画面校正.提出一种颜色校正方案以解决上述问题.首先介绍多投影显示墙画面校正中存在的主要问题和现有的解决方法;在此基础上,给出了多投影显示墙的画面校正流程和几何校正方法;然后讨论了多投影显示墙画面校正结果的评价标准,提出了投影仪的广义颜色模型和基于搜索技术的视觉无缝方法.该方案使用数码相机实现了高精度的视觉无缝画面校正,从而提高了构造多投影显示墙的效率并降低了维护成本.校正结果在多种类型的投影墙系统中得到验证和应用,对指导多投影显示墙的搭建和维护具有较高的理论与实践价值.     

Introduction to building projection-based tiled display systems

This tutorial introduces the concepts and technologies needed to build projector-based display systems. Tiled displays offer scalability, high resolution, and large formats for various applications. Tiled displays are an emerging technology for constructing semi-immersive visualization environments capable of presenting high-resolution images from scientific simulation. The largest impact may well arise from using large-format tiled displays as one of possibly multiple displays in building information or active spaces that surround the user with diverse ways of interacting with data and multimedia information flows. These environments may prove the ultimate successor to the desktop metaphor for information technology work. Several fundamental technological problems must be addressed to make tiled displays practical. These include: the choice of screen materials and support structures; choice of projectors, projector supports, and optional fine positioners; techniques for integrating image tiles into a seamless whole; interface devices for interaction with applications; display generators and interfaces; and the display software environment     

Tele-immersive collaborative virtual environment for intuitive interpretation

The purpose of tele-immersive collaboration is to enable people in separate locations to work together effectively by the Virtual Reality (VR) technology facilitating the human intuitive insight and interpretation. Consequently, it is important to cogitate with figures and characters as well as voice communication in Collaborative Virtual Environment (CVE). In this paper, we constructed tele-immersive CVE with annotation capability and free-hand drawing of characters and figures into Immersive Projection Technology (IPT) environment. In addition, participants from remote sites connected over a Wide Area Network (WAN) were enabled to interact with each other by sharing annotations via a PDA application. Moreover, we developed applications for scientific visualization and, for intellectual work based on KJ Method inside a tele-immersive CVE. From these results, we are able to maintain essential information for thinking by making annotation on IPT environment, and have proved the system can support intuitive understanding in remote collaborative work. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007     

多投影显示墙画面校正技术综述*

由投影仪阵列组成的多投影显示墙是目前实现高性能、低成本大型绘制平台的首选解决方案。为了保证多投影显示墙画面的完整性,需要通过一定的方法来预补偿由于投影仪特性差异造成的画面几何扭曲与色彩不均衡。结合实际研究工作,从多投影显示墙存在的问题、几何变换矩阵求取、颜色模型估计和全局颜色校正等几个方面对多投影显示墙画面校正的研究情况进行综述, 讨论了存在的问题和进一步的发展方向。     

The perceptual scalability of visualization

Larger, higher resolution displays can be used to increase the scalability of information visualizations. But just how much can scalability increase using larger displays before hitting human perceptual or cognitive limits? Are the same visualization techniques that are good on a single monitor also the techniques that are best when they are scaled up using large, high-resolution displays? To answer these questions we performed a controlled experiment on user performance time, accuracy, and subjective workload when scaling up data quantity with different space-time-attribute visualizations using a large, tiled display. Twelve college students used small multiples, embedded bar matrices, and embedded time-series graphs either on a 2 megapixel (Mp) display or with data scaled up using a 32 Mp tiled display. Participants performed various overview and detail tasks on geospatially-referenced multidimensional time-series data. Results showed that current designs are perceptually scalable because they result in a decrease in task completion time when normalized per number of data attributes along with no decrease in accuracy. It appears that, for the visualizations selected for this study, the relative comparison between designs is generally consistent between display sizes. However, results also suggest that encoding is more important on a smaller display while spatial grouping is more important on a larger display. Some suggestions for designers are provided based on our experience designing visualizations for large displays.     

Evaluation study on realistic sensation in tele-communication environment with ultra-resolution video by multiple cameras on tiled display wall

To enhance a high-realistic sensation for participants in tele-communication over the Internet, space sharing by displaying super-quality video is effective. However, in commonly used tele-communication systems, participants have found it difficult to feel an ordinary presence and realistic sensation in real environments, and an effective solution has still not been proposed. We have constructed a tele-communication environment to display an ultra-resolution video which realizes a display resolution over 4K on a tiled display wall by multi-transmitting each video captured by multiple cameras. Nonetheless, the need for objective evaluation and analysis on the realistic sensation that participants feel in tele-communication has been increasing. However, there are only a few case studies on detailed analysis and evaluation of the factors required for realistic sensation in tele-communication. In this paper, we have conducted an experiment to evaluate and analyze the effectiveness of realistic sensation and its effect on participants in this tele-communication environment. From experimental results, we have shown that this tele-communication environment has a strong tendency to provide higher realistic sensation for estimators.     

多投影仪无缝拼接显示中的颜色校正技术比较研究

基于多投影仪的无缝拼接显示是对图形、图像、视频等进行大范围、高分辨率显示的一种有效实现方式。多投影仪无缝拼接显示的一个关键是解决颜色失调问题,而现有的颜色失调原因的分析结果可归纳为投影仪输出颜色的变化特性和投影屏幕与投影环境的影响两个方面。近年来,为解决该问题已提出了多种颜色校正技术,而根据解决颜色失调问题的原理和实现方法的不同,这些技术可分为基于边缘融合的校正技术、基于单一投影光源的校正技术和基于颜色/亮度输出匹配的校正技术3类,并对这3类技术在无缝拼接显示效果、可维护性和可扩展性等方面的优缺点进行了较详细分析,该领域下一步的研究将会朝着兼顾不同投影机类型、不同形状和反射特性的投影屏幕和考虑运动中观察者的实时颜色校正方向发展。     

Large high resolution displays for co-located collaborative sensemaking: Display usage and territoriality

From an exploratory user study using a fictional textual intelligence analysis task on a large, high-resolution vertical display, we investigated how pairs of users interact with the display to construct spatial schemas and externalize information, as well as how they establish shared and private territories. We investigated users' space management strategies depending on the design philosophy of the user interface (visualization- or document-centric). We classified the types of territorial behavior exhibited in terms of how the users interacted with information on the display (integrated or independent workspaces). Next, we examined how territorial behavior impacted the common ground between the pairs of users. Finally, we offer design suggestions for building future co-located collaborative visual analytics tools for use on large, high-resolution vertical displays.     

背投式多投影系统中各向异性问题的研究

目的 针对背投式多投影系统中存在的各向异性问题,提出一套行之有效的解决方案.方法 首先,为了解决以静态观众为主体的多投影系统各向异性问题,基于反馈机制提出了以特定位置为中心的“最佳观测域”,并在该区域内实现了真正的视觉无缝画面校正;然后,基于视觉人体跟踪技术进一步提出了以特定交互者为中心的“动态最佳观测域”框架,并结合模糊预测控制思想从人体重心检测、跟踪和投影仪亮度实时补偿等3个方面对该框架内的各个组成模块的关键技术进行了阐述.结果 实验结果显示,利用交互者运动的时空连续性优化后的动态最佳观测域算法,具有较好的鲁棒性和实时性.同时,与其他类似方法相比获得了更好的视觉无缝校正效果,其中观测距离为1 m、3 m和5 m时,对应的亮度样本标准差分别为17.11、13.17和9.2.结论 提出的各向异性解决方案,将实时反馈信号引入校正过程中,有效提高了输出画面质量.通过实验测试和性能评估,验证了文中方法的可行性.     

A unified paradigm for scalable multi-projector displays

We present a general framework for the modeling and optimization of scalable multi-projector displays. Based on this framework, we derive algorithms that can robustly optimize the visual quality of an arbitrary combination of projectors without manual adjustment. When the projectors are tiled, we show that our framework automatically produces blending maps that outperform state-of-the-art projector blending methods. When all the projectors are superimposed, the framework can produce high-resolution images beyond the Nyquist resolution limits of component projectors. When a combination of tiled and superimposed projectors are deployed, the same framework harnesses the best features of both tiled and superimposed multi-projector projection paradigms. The framework creates for the first time a new unified paradigm that is agnostic to a particular configuration of projectors yet robustly optimizes for the brightness, contrast, and resolution of that configuration. In addition, we demonstrate that our algorithms support high resolution video at real-time interactive frame rates achieved on commodity graphics platforms. This work allows for inexpensive, compelling, flexible, and robust large scale visualization systems to be built and deployed very efficiently.     

大规模立体显示墙系统的构建

从硬件选择、几何校正、边缘融合、颜色校正和软件环境等5方面系统地研究了构建大规模立体显示墙系统的关键技术.采用并行绘制技术和基于计算机视觉的自动校正,由Linux集群驱动多台普通投影仪拼接成一个统一的高分辨率、大规模的被动立体显示系统.     

桌面环境下拼贴显示的自动对准

通过拼贴一组投影仪来构建高分辨率显示系统已成为现在一个较实际的办法.但是,这样的高分辨率显示系统用途有限,因为它们需要在定制的并行机或个人计算机集群上运行一些专门开发的并行形象化程序才能实现.针对个人计算机桌面环境下的任意拼贴显示给出了一种自动对准机制,使得桌面用户可以在其上运行普通的桌面应用软件.该系统包括3个步骤:检测投影仪对齐失准,计算出纠正所需之变换,对桌面环境进行实时的变形.这样就允许用户在运行任何2D,3D或视频程序时无须作任何修改,也无须使用专门的硬件支持.实验结果表明,系统能够获得亚像素级的精度,并且能在系统性能衰减最小的情况下达到实时变形.     

基于Web集群的海量影像显示技术研究

针对目前Web显示技术只允许单个用户通过远程视频会议和桌面镜像与单个异地用户协同工作,不能满足多用户海量信息共享显示的要求,不支持海量影像超高分辨显示需要等问题.本文根据多用户海量信息共享显示的特点以及显示的难点,研究了基于Web集群海量影像多异地用户在拼接显示屏上超高分辨显示等技术.实验结果表明,提出的Web集群并行拼接海量影像显示技术是有效的,能够同时为多个异地用户的海量影像在拼接显示屏上超高分辨显示提供可行的解决方案.     

基于机群的拼贴显示系统的软件结构

利用计算机系统控制多个图形部件,驱动多个投影仪得到基于投影拼贴的显示,是突破现有显示技术制约,实现千万像素以上的高清晰度大尺寸显示的有效手段。目前采用PC机群系统来驱动拼贴显示系统的方式由于其较高的性能价格比正日益成为主流的选择,但是这种实现方式同时需要更复杂的分布式显示软件的支持。该文分析比较了几种基于机群的拼贴显示系统的软件结构并在此基础上提出了一种新的适合并行程序环境的软件接口。     

Meme Media for Clipping and Combining Web Resources

The publication and reuse of intellectual resources using the Web technologies provide no support for us to clip out any portion of Web pages, to combine them together for their local reuse, nor to publish the newly composed object as a new Web page for its reuse by other people. This paper shows how the meme-media architecture is applied to the Web to provide such support for us. This makes the Web work as a shared repository not only for publishing intellectual resources, but also for their collaborative reediting. We will propose a general framework for clipping arbitrary Web contents as live objects, for defining IO ports on such a clip, and for the recombination and linkage of such clips based on both the original and some user-defined relationships among them. In our previous works, we proposed two separate frameworks for these three purposes; one works for the first two, and the other for the last. Here we will propose a unified framework for these three purposes, as well as its detailed internal mechanisms. Then we show how it can be easily applied to various legacy Web applications to develop innovative services.     

Remote collaboration on desk‐sized displays

Work on remote collaboration has often focused on the person space created by a conventional videoconference where the participants see each other's faces, but we argue that a task space containing shared visual information is more important for most tasks. Trends in display technology mean that large visual task spaces can be created to maximize the shared context between collaborators. We have created a system called the Escritoire that presents users with a desk‐sized projected display with bimanual input that allows documents and images to be arranged and modified by multiple remote collaborators. We describe the software architecture, the protocol that is used between the client and server programs, and the pen traces we have added to allow participants to gesture to each other in the large visual space to enrich their communication. Our user trials have shown that participants were able to use the system with a minimum of training, and found the traces useful in the collaborative setting. In future we will be connecting three or more sites together to explore the issues that arise with multi‐party interaction on large shared desks. Copyright © 2005 John Wiley & Sons, Ltd.