Real-time ball-position measurement using multi-view cameras for live football broadcast

Multimedia Tools and Applications - We developed a robust ball-position-measurement system that can be used for actual football games. It measures the positions of a ball by integrating tracking...     

Real-time structured light coding for adaptive patterns

Coded structured light is a technique that allows the 3D reconstruction of poorly or non-textured scene areas. With the codes uniquely associated with visual primitives of the projected pattern, the correspondence problem is quickly solved by means of local information only, with robustness against disturbances like high surface curvatures, partial occlusions, out-of-field of view or out-of-focus. Real-time 3D reconstruction with one shot is possible with pseudo-random arrays, where the encoding is done in a single pattern using spatial neighbourhood. To correct more mismatched visual primitives and to get patterns globally more robust, a higher Hamming distance between all the used codewords should be suited. Recent works in the structured light field have shown a growing interest for adaptive patterns. These can account for geometrical or spectral specificities of the scene to provide better features matching and reconstructions. Up till today, such patterns cannot benefit from the robustness offered by spatial neighbourhood coding with a minimal Hamming distance constraint, because the existing algorithms for such a class of coding are designed with an offline coding only. In this article, we show that due to two new contributions, a mixed exploration/exploitation search behaviour and a O(n ²) to ～O(n) complexity reduction using the epipolar constraint, the real-time coding of patterns having similar properties than those coded offline can be achieved. This allows to design a complete closed-loop processing pipeline for adaptive patterns.     

Real-time content filtering for live broadcasts in TV terminals

In this paper, we propose a new real-time content filtering framework for live broadcasts in TV terminals. Content filtering in TV terminals is a necessary provision of personalized broadcasting services in that it enables a TV viewer to obtain desired scenes from multiple channel broadcasts. In this paper, a stable and reliable filtering structure and an algorithm for multiple inputs are proposed. Moreover, real-time filtering requirements such as frame sampling rate per channel, number of input channels, and buffer condition are analyzed to achieve real-time processing in terminals with limited computing power. Based on queueing theory, we model the system and resolve the filtering requirements. To verify the proposed system and analysis, a filtering algorithm for soccer videos is applied which is modified for real-time processing. Through analysis of visual features (e.g., dominant color and edge components) and detection of spatial objects (e.g., a score board), it recognizes a temporal pattern between successive video frames and filters desired scenes. Experiments on soccer videos have been performed and the results validate the effectiveness of the proposed approach and system.

Real-time range acquisition by adaptive structured light

The goal of this paper is to provide a "self-adaptive" system for real-time range acquisition. Reconstructions are based on a single frame structured light illumination. Instead of using generic, static coding that is supposed to work under all circumstances, system adaptation is proposed. This occurs on-the-fly and renders the system more robust against instant scene variability and creates suitable patterns at startup. A continuous trade-off between speed and quality is made. A weighted combination of different coding cues - based upon pattern color, geometry, and tracking - yields a robust way to solve the correspondence problem. The individual coding cues are automatically adapted within a considered family of patterns. The weights to combine them are based on the average consistency with the result within a small time-window. The integration itself is done by reformulating the problem as a graph cut. Also, the camera-projector configuration is taken into account for generating the projection patterns. The correctness of the range maps is not guaranteed, but an estimation of the uncertainty is provided for each part of the reconstruction. Our prototype is implemented using unmodified consumer hardware only and, therefore, is cheap. Frame rates vary between 10 and 25 fps, dependent on scene complexity.     

Real-time per-pixel focusing method for light field rendering

Computational Visual Media - Light field rendering is an image-based rendering method that does not use 3D models but only images of the scene as input to render new views. Light field...     

Real-time display and development of an aerometric data base for a field experiment

An innovative data center responsible for collecting, displaying, and analyzing near real-time meteorological and air quality measurements from a variety of sources was operated in Camarillo, California, during September–October 1985. This data center was part of a multi-million dollar study conducted for the South Central Coast Cooperative Monitoring Program sponsored by the Western Oil and Gas Association, a consortium of petroleum and fuel-producing organizations, to gain a better understanding of air quality conditions in the southern California coastal areas, and to produce a comprehensive data base for which coastal air quality models could be developed and evaluated.Led by the Atmospheric Science Center of SRI International, the program included a large number of contractor organizations responsible for collecting air quality measurements from land, sea, and air instrumentation for future analysis and model development. The data center integrated near real-time air quality and meteorological data with weather services forecasts to develop day-to-day environmental predictions governing the data collection activities.     

自适应嵌入的半监督多视角特征降维方法

半监督模式下的多视角特征降维方法,大多并未考虑到不同视角间特征投影的差异,且由于缺乏对降维后的低维矩阵的稀疏约束,无法避免噪声和其他不相关特征的影响。针对这两个问题,提出自适应嵌入的半监督多视角特征降维方法。首先,将投影从单视角下相同的嵌入矩阵扩展到多视角间不同的矩阵,引入全局结构保持项;然后,将无标签的数据利用无监督方法进行嵌入投影,对于有标签的数据,结合分类的判别信息进行线性投影;最后,再将两类多投影映射到统一的低维空间,使用组合权重矩阵来保留全局结构,很大程度上消除了噪声及不相关因素的影响。实验结果表明,所提方法的聚类准确率平均提高了约9%。该方法较好地保留了多视角间特征的相关性,捕获了更多的具有判别信息的特征。     

立体液晶显示器的图像获取及显示

通过研究立体液晶显示器的图像获取及显示接口,完成了从摄像到显像的整体数字系统的电路部分设计。     

Wide field of view compressive light field display using a multilayer architecture and tracked viewers

In this paper, we discuss an intuitive extension to compressive multilayer light field displays that greatly extends their field of view and depth of field. Rather than optimizing these displays to create a moderately narrow field of view at the center of the display, we constrain optimization to create narrow view cones that are directed to a few viewers' eyes, allowing the available display bandwidth to be utilized more efficiently. These narrow view cones follow the viewers, creating a wide apparent field of view. Imagery is also recalculated for the viewers' exact eye positions, creating a greater depth of field. The view cones can be scaled to match the positional error and latency of the tracking system. Using more efficient optimization and commodity tracking hardware and software, we demonstrate a real‐time, glasses‐free 3D display that offers a 100° × 40° field of view.     

视频实时自适应去雾算法

在雾天环境下,户外视频的可视性将受到极大损害,需要通过视频实时去雾来恢复视频的可视性。视频实时去雾对于单帧图像处理的速度有很高的要求,现有的图像去雾算法或是速度上达不到要求,或是速度虽快但去雾效果不理想。另外,视频还会面临拍摄场景中雾气浓度不断变化的问题,现有图像去雾算法中需要手动设置参数且参数固定,无法在雾气浓度变化的条件下始终达到理想的去雾效果。提出了一种实时的视频自适应去雾算法,该算法对视频中单帧图像进行去雾时,会基于暗原色值来区分图像区域,并对不同区域进行不同程度的去雾,在满足实时性的同时得到了很好的去雾效果。此外,该算法还基于暗通道先验设计了评价去雾结果的方法,并使用迭代的方式根据雾气浓度自动调整去雾参数,从而在视频中雾气浓度变化的情况下,始终能达到理想的去雾效果。     

An optimization method for parameters configuration of the light field display based on subjective evaluation

A light field display (LFD) can provide high immersion three-dimensional (3D) visual experience by accurately reconstructing the light field information of 3D objects, which usually require a large number of views and high resolution. However, the information capacity provided by the LFD is limited, so the view number and the view resolution are mutually constrained. Few effective standards or methods have been proposed to guide the configuration of the view number and the resolution, which can only be determined empirically. Here, an optimization method for parameters configuration of the LFD is proposed with four subjective experiments. The display quality of the LFD with different configurations between the view number and the view resolution is quantitatively evaluated. Accordingly, a physiological visual model (PVM) is established to depict the change rules and guide the parameters configuration. Experimental results demonstrate the validity of the proposed optimization method quantitatively and qualitatively, which will contribute to the researches of the LFD in the future.     

A multi-view attention-based deep learning system for online deviant content detection

With the exponential growth of user-generated content, policies and guidelines are not always enforced in social media, resulting in the prevalence of deviant content violating policies and guidelines. The adverse effects of deviant content are devastating and far-reaching. However, the detection of deviant content from sparse and imbalanced textual data is challenging, as a large number of stakeholders are involved with different stands and the subtle linguistic cues are highly dependent on complex context. To address this problem, we propose a multi-view attention-based deep learning system, which combines random subspace and binary particle swarm optimization (RS-BPSO) to distill content of interest (candidates) from imbalanced data, and applies the context and view attention mechanisms in convolutional neural network (dubbed as SSCNN) for the extraction of structural and semantic features. We evaluate the proposed approach on a large-scale dataset collected from Facebook, and find that RS-BPSO is able to detect whether the content is associated with marijuana with an accuracy of 87.55%, and SSCNN outperforms baselines with an accuracy of 94.50%.

     

传感器网络中一种实时的自适应时隙调度算法

无线传感器网络节点数目众多,MAC协议为节点分配工作时隙面临能量利用不高、节点延时较长等方面的难题。目前基于时隙调度的MAC协议一般采用等长的时隙大小,不能适应数据流量变化大的网络且忽略与网络层的融合,没有利用路由层信息来减低时隙分配算法性能代价。提出一种基于路由转发树的时隙调度算法（ATSA）,网络采用簇结构,在簇内构造一棵路由转发树,根据路由转发树形成的路径信息对节点实时获取节点每轮需要发送的数据量大小,根据节点的数据量大小来分配节点每轮需要的时隙,然后由簇头据此动态地为成员节点分配时隙,降低时隙划分的能量和时间代价,减少空闲侦听时间,避免串音。仿真表明,该算法有效地提高了网络能量利用效率,延长了网络生存周期,降低数据包的延时。     

基于网络抓包的数据库即时备份方法

为了适应各类数据库的即时备份,提出了基于捕捉网络数据包模式的数据库即时备份方法.该方法不需要在数据库服务器上进行配置或安装软件;以网络抓包技术实时捕捉数据库服务器数据包,通过分析数据包整理出数据库写语句,并形成写语句集;最后把写语句集中的SQL语句依次发往备份数据库,实现数据库的即时备份.实验结果表明,该方法备份实时性较高.     

Toward the light field display: autostereoscopic rendering via a cluster of projectors

Ultimately, a display device should be capable of reproducing the visual effects observed in reality. In this paper we introduce an autostereoscopic display that uses a scalable array of digital light projectors and a projection screen augmented with microlenses to simulate a light field for a given three-dimensional scene. Physical objects emit or reflect light in all directions to create a light field that can be approximated by the light field display. The display can simultaneously provide many viewers from different viewpoints a stereoscopic effect without head tracking or special viewing glasses. This work focuses on two important technical problems related to the light field display; calibration and rendering. We present a solution to automatically calibrate the light field display using a camera and introduce two efficient algorithms to render the special multi-view images by exploiting their spatial coherence. The effectiveness of our approach is demonstrated with a four-projector prototype that can display dynamic imagery with full parallax.     

An interactive 3D medical visualization system based on a light field display

We present a prototype medical data visualization system exploiting a light field display and custom direct volume rendering techniques to enhance understanding of massive volumetric data, such as CT, MRI, and PET scans. The system can be integrated with standard medical image archives and extends the capabilities of current radiology workstations by supporting real-time rendering of volumes of potentially unlimited size on light field displays generating dynamic observer-independent light fields. The system allows multiple untracked naked-eye users in a sufficiently large interaction area to coherently perceive rendered volumes as real objects, with stereo and motion parallax cues. In this way, an effective collaborative analysis of volumetric data can be achieved. Evaluation tests demonstrate the usefulness of the generated depth cues and the improved performance in understanding complex spatial structures with respect to standard techniques.

Real-time 3D human capture system for mixed-reality art and entertainment

A real-time system for capturing humans in 3D and placing them into a mixed reality environment is presented in this paper. Nine cameras surrounding her capture the subject. Looking through a head-mounted-display with a camera in front pointing at a marker, the user can see the 3D image of this subject overlaid onto a mixed reality scene. The 3D images of the subject viewed from this viewpoint are constructed using a robust and fast shape-from-silhouette algorithm. The paper also presents several techniques to produce good quality and speed up the whole system. The frame rate of our system is around 25 fps using only standard Intel processor-based personal computers. Besides a remote live 3D conferencing and collaborating system, we also describe an application of the system in art and entertainment, named Magic Land, which is a mixed reality environment where captured avatars of human and 3D computer generated virtual animations can form an interactive story and play with each other. This system demonstrates many technologies in human computer interaction: mixed reality, tangible interaction, and 3D communication. The result of the user study not only emphasizes the benefits, but also addresses some issues of these technologies.