EISEval,a generic reconfigurable environment for evaluating agent-based interactive systems

The evaluation of interactive systems has been an active subject of research for many years. Many methods have been proposed, but most of them do not take the architectural specificities of an agent-based interactive system into account, nor do they focus on the link between architecture and evaluation. In this paper, we present an agent-based architecture model for interactive systems. Then, based on this architecture, we propose a generic, reconfigurable evaluation environment, called EISEval, designed and developed to help evaluators analyze and evaluate certain aspects of interactive systems in general and of agent-based architecture interactive systems in particular: User Interface (UI), non-functional properties (e.g., response time, complexity) and user characteristics (e.g., abilities, preferences, progress). System designers can draw useful conclusions from the evaluation results to improve the system. This environment was applied to evaluate an agent-based interactive system used to supervise an urban transport network in a study organized in laboratory.     

Augmenting Tactile 3D Data Navigation With Pressure Sensing

We present a pressure‐augmented tactile 3D data navigation technique, specifically designed for small devices, motivated by the need to support the interactive visualization beyond traditional workstations. While touch input has been studied extensively on large screens, current techniques do not scale to small and portable devices. We use phone‐based pressure sensing with a binary mapping to separate interaction degrees of freedom (DOF) and thus allow users to easily select different manipulation schemes (e. g., users first perform only rotation and then with a simple pressure input to switch to translation). We compare our technique to traditional 3D‐RST (rotation, scaling, translation) using a docking task in a controlled experiment. The results show that our technique increases the accuracy of interaction, with limited impact on speed. We discuss the implications for 3D interaction design and verify that our results extend to older devices with pseudo pressure and are valid in realistic phone usage scenarios.     

Determinants of e-consumer productivity in product retrieval on a commercial website: An experimental approach

This article investigates what determines e-consumer productivity, in the specific case of product retrieval, on a commercial website. With a 2 × 2 × 2 factorial design on 292 participants, an online experiment reveals that productivity in product retrieval (measured in terms of effectiveness, efficiency, and time) relates to website design (e.g., abstraction level of labels, animation), user characteristics (e.g., Internet experience, product category familiarity, cognitive absorption), and situational characteristics (e.g., task nature). The results also confirm interactive effects among the type of strategy used, the nature of the task, and the website design. These findings have notable implications for both research and practice.     

基于深度学习的视觉目标检测技术综述

视觉目标检测旨在定位和识别图像中存在的物体,属于计算机视觉领域的经典任务之一,也是许多计算机视觉任务的前提与基础,在自动驾驶、视频监控等领域具有重要的应用价值,受到研究人员的广泛关注。随着深度学习技术的飞速发展,目标检测取得了巨大的进展。首先,本文总结了深度目标检测在训练和测试过程中的基本流程。训练阶段包括数据预处理、检测网络、标签分配与损失函数计算等过程,测试阶段使用经过训练的检测器生成检测结果并对检测结果进行后处理。然后,回顾基于单目相机的视觉目标检测方法,主要包括基于锚点框的方法、无锚点框的方法和端到端预测的方法等。同时,总结了目标检测中一些常见的子模块设计方法。在基于单目相机的视觉目标检测方法之后,介绍了基于双目相机的视觉目标检测方法。在此基础上,分别对比了单目目标检测和双目目标检测的国内外研究进展情况,并展望了视觉目标检测技术发展趋势。通过总结和分析,希望能够为相关研究人员进行视觉目标检测相关研究提供参考。     

Real-time hand detection using continuous skeletons

In this paper, a fast and reliable method for hand detection based on continuous skeletons approach is presented. It demonstrates real-time working speed and high detection accuracy (3–5% both FAR and FRR) on a large dataset (50 persons, 80 videos, 2322 frames). These make it suitable for use as a part of modern hand identification systems including mobile ones. Overall, the study shows that continuous skeletons approach can be used as prior for object and background color models in segmentation methods with supervised learning (e.g., interactive segmentation with seeds or abounding box).     

Exemplar-based statistical model for semantic parametric design of human body

This paper presents an exemplar-based method to provide intuitive way for users to generate 3D human body shape from semantic parameters. In our approach, human models and their semantic parameters are correlated as a single linear system of equations. When users input a new set of semantic parameters, a new 3D human body will be synthesized from the exemplar human bodies in the database. This approach involves simpler computation compared to non-linear methods while maintaining quality outputs. A semantic parametric design in interactive speed can be implemented easily. Furthermore, a new method is developed to quickly predict whether the parameter values is reasonable or not, with the training models in the human body database. The reconstructed human bodies in this way will all have the same topology (i.e., mesh connectivity), which facilitates the freeform design automation of human-centric products.     

基于Leap Motion的三维手势交互系统研究

手势识别的快速发展及体感设备的不断更新为三维手势交互提供了灵感,基于Leap Motion 手势识别和最邻近算法,建立了一种三维手势交互系统。首先对手势设计理论和交互手 势设计原则进行研究,基于此设计手势功能和建立手势库,并将手势库分为 8 种手势;其次进 行手势特征提取,建立手指关键点模型,获取手势特征的角度特征;然后计算 KNN 算法和 SVM 算法的手势识别效率,KNN 改进算法取得较好的识别效率;最后,设计三维交互系统,手势分 类为 4 个模块,每个模块有 2 个手势任务;20 名测试者中提取 1 600 组手势数据,并进行总采 集样本关节点均值的数据分析;设计三维交互系统模块,在 Unity3D 中创建的三维交互系统中 导入 1 600 组手势数据,根据自定义的 8 种手势驱动虚拟手完成交互设计过程,完成用户体验 分析和手势识别效率统计。通过研究发现,基于 Leap Motion 手势识别具有较高的识别效率, 三维手势交互系统富有创新性。     

Animated visualization of spatial–temporal trajectory data for air-traffic analysis

With increasing numbers of flights worldwide and a continuing rise in airport traffic, air-traffic management is faced with a number of challenges. These include monitoring, reporting, planning, and problem analysis of past and current air traffic, e.g., to identify hotspots, minimize delays, or to optimize sector assignments to air-traffic controllers. To cope with these challenges, cyber worlds can be used for interactive visual analysis and analytical reasoning based on aircraft trajectory data. However, with growing data size and complexity, visualization requires high computational efficiency to process that data within real-time constraints. This paper presents a technique for real-time animated visualization of massive trajectory data. It enables (1) interactive spatio-temporal filtering, (2) generic mapping of trajectory attributes to geometric representations and appearance, and (3) real-time rendering within 3D virtual environments such as virtual 3D airport or 3D city models. Different visualization metaphors can be efficiently built upon this technique such as temporal focus+context, density maps, or overview+detail methods. As a general-purpose visualization technique, it can be applied to general 3D and 3+1D trajectory data, e.g., traffic movement data, geo-referenced networks, or spatio-temporal data, and it supports related visual analytics and data mining tasks within cyber worlds.     

基于拓扑分析的三维动漫人物造型重构

传统方法在三维动漫人物造型重构过程中未确定关键特征点,导致姿态图像匹配度低和收敛速度慢,为此设计一种基于拓扑分析的三维动漫人物造型重构方法。引用拓扑分析确定三维动漫人物造型的点、线、面结构,运用基于融合技术的边缘检测方法提取造型特征点,计算特征点的梯度值和梯度方向,以此来确定关键特征点,生成主方向和特征描述符。在此基础上计算与其对应的源数据库中关键点的描述符距离,根据最小距离和次小距离的比值实现特征点的匹配,利用交互式几何约束变形完成三维动漫人物造型重构。测试结果表明：设计的三维动漫人物造型重构方法的姿态图像匹配度整体在0.6以上,远高于传统重构方法的匹配度,且该方法的收敛速度较快,说明其适合应用在三维动漫人物造型重构设计中。     

Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction

Robotic teleoperation, i.e., manipulating remote robotic systems at a distance, has gained its popularity in various industrial applications, including construction operations. The key to a successful teleoperation robot system is the delicate design of the human-robot interface that helps strengthen the human operator’s situational awareness. Traditional human-robot interface for robotic teleoperation is usually based on imagery data (e.g., video streaming), causing the limited field of view (FOV) and increased cognitive burden for processing additional spatial information. As a result, 3D scene reconstruction methods based on point cloud models captured by scanning technologies (e.g., depth camera and LiDAR) have been explored to provide immersive and intuitive feedback to the human operator. Despite the added benefits of applying reconstructed 3D scenes in telerobotic systems, challenges still present. Most 3D reconstruction methods utilize raw point cloud data due to the difficulty of real-time model rendering. The significant size of point cloud data makes the processing and transfer between robots and human operators difficult and slow. In addition, most reconstructed point cloud models do not contain physical properties such as weight and colliders. A more enriched control mechanism based on physics engine simulations is impossible. This paper presents an intelligent robot teleoperation interface that collects, processes, transfers, and reconstructs the immersive scene model of the workspace in Virtual Reality (VR) and enables intuitive robot controls accordingly. The proposed system, Telerobotic Operation based on Auto-reconstructed Remote Scene (TOARS), utilizes a deep learning algorithm to automatically detect objects in the captured scene, along with their physical properties, based on the point cloud data. The processed information is then transferred to the game engine where rendered virtual objects replace the original point cloud models in the VR environment. TOARS is expected to significantly improve the efficiency of 3D scene reconstruction and situational awareness of human operators in robotic teleoperation.     

基于结构先验与协同优化的城市场景分段平面重建

在基于图像的城市场景三维重建中,场景分段平面重建算法可以克服场景中的弱纹理、光照变化等因素的影响而快速恢复场景完整的近似结构.然而,在初始空间点较为稀疏、候选平面集不完备、图像过分割质量较低等问题存在时,可靠性往往较低.为了解决此问题,本文根据城市场景的结构特征构造了一种新颖的融合场景结构先验、空间点可见性与颜色相似性的平面可靠性度量,然后采用图像区域与相应平面协同优化的方式对场景结构进行了推断.实验结果表明,本文算法利用稀疏空间点即可有效重建出完整的场景结构,整体上具有较高的精度与效率.     

脉冲视觉研究进展

视频是视觉信息处理的基础概念,传统视频的帧率只有几十Hz,不能记录光的高速变化过程,成为限制机器视觉速度的天花板,其根本原因在于视频概念脱胎于胶片成像,未能发挥电子和数字技术的潜力。脉冲视觉模型通过感光器件捕获光子,累积能量达到约定阈值时产生脉冲,形成脉冲的时间越长,表明收到的光信号越弱,反之光信号越强,据此可估计任意时刻的光强,从而实现连续成像。采用普通器件,研制了比影视视频快千倍的超高速成像芯片和相机,进而基于脉冲神经网络实现了超高速目标检测、跟踪和识别,打破了机器视觉提速依赖算力线性增长的传统范式。本文从脉冲视觉模型表达视觉信息的生物学基础和物理原理出发,介绍了脉冲视觉原理的软件模拟器及其模拟真实世界光子传播的计算过程,描述了基于脉冲视觉原理的高灵敏光电传感器件及芯片的工作机理和结构设计、基于脉冲视觉的影像重建原理以及脉冲视觉信号与普通图像信号融合的计算摄像算法与计算摄像系统,介绍了基于脉冲神经网络的超高速运动目标检测、跟踪与识别,通过对比国际国内相关研究内容和发展现状,展望了脉冲视觉的发展与演进方向。脉冲视觉芯片和系统在工业(高铁、电力和轮机等不停机监测,智能制造高速监视等)、民用(高速相机、智能交通、辅助驾驶、司法取证和体育判罚等)以及国防(高速对抗)等领域都具有巨大应用潜力,是未来值得重点关注和研究的一个重要方向。     

Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video

3D video [IEEE Multimedia (1997) 18] is the ultimate image media recording dynamic visual events in the real world as is; it records time varying 3D object shape with high fidelity surface properties (i.e., color and texture). Its applications cover wide varieties of personal and social human activities: entertainment (e.g., 3D game and 3D TV), education (e.g., 3D animal picture books), sports (e.g., sport performance analysis), medicine (e.g., 3D surgery monitoring), culture (e.g., 3D archive of traditional dances), and so on. In this paper, we propose: (1) a PC cluster system for real-time reconstruction of dynamic 3D object action from multi-view video images, (2) a deformable 3D mesh model for reconstructing the accurate dynamic 3D object shape, and (3) an algorithm of rendering natural-looking texture on the 3D object surface from the multi-view video images. Experimental results with quantitative performance evaluations demonstrate the effectiveness of these methods in generating high fidelity 3D video from multi-view video images.     

基于小波的多分辨率贴面算法及实现

贴面是指在两个平行截面的轮廓线间用面片（通常是三角面片）贴补侧面,从而得到一个三维实体的过程,贴面是三维重建的一项内容,有重要的应用价值,例如医学领域由人体器官切片图来重建器官三维模型,地形方面由等高线得到地形等。本文介绍多分辨率贴面算法及其实现技术,并给出了运行实例。     

Analysis of Long Molecular Dynamics Simulations Using Interactive Focus+Context Visualization

Analyzing molecular dynamics (MD) simulations is a key aspect to understand protein dynamics and function. With increasing computational power, it is now possible to generate very long and complex simulations, which are cumbersome to explore using traditional 3D animations of protein movements. Guided by requirements derived from multiple focus groups with protein engineering experts, we designed and developed a novel interactive visual analysis approach for long and crowded MD simulations. In this approach, we link a dynamic 3D focus+context visualization with a 2D chart of time series data to guide the detection and navigation towards important spatio‐temporal events. The 3D visualization renders elements of interest in more detail and increases the temporal resolution dependent on the time series data or the spatial region of interest. In case studies with different MD simulation data sets and research questions, we found that the proposed visual analysis approach facilitates exploratory analysis to generate, confirm, or reject hypotheses about causalities. Finally, we derived design guidelines for interactive visual analysis of complex MD simulation data.     

Priority-Driven Acoustic Modeling for Virtual Environments

Geometric acoustic modeling systems spatialize sounds according to reverberation paths from a sound source to a receiver to give an auditory impression of a virtual 3D environment. These systems are useful for concert hall design, teleconferencing, training and simulation, and interactive virtual environments. In many cases, such as in an interactive walkthrough program, the reverberation paths must be updated within strict timing constraints - e.g., as the sound receiver moves under interactive control by a user. In this paper, we describe a geometric acoustic modeling algorithm that uses a priority queue to trace polyhedral beams representing reverberation paths in best-first order up to some termination criteria (e.g., expired time-slice). The advantage of this algorithm is that it is more likely to find the highest priority reverberation paths within a fixed time-slice, avoiding many geometric computations for lower-priority beams. Yet, there is overhead in computing priorities and managing the priority queue. The focus of this paper is to study the trade-offs of the priority-driven beam tracing algorithm with different priority functions. During experiments computing reverberation paths between a source and a receiver in a 3D building environment, we find that priority functions incorporating more accurate estimates of source-to-receiver path length are more likely to find early reverberation paths useful for spatialization, especially in situations where the source and receiver cannot reach each other through trivial reverberation paths. However, when receivers are added to the environment such that it becomes more densely and evenly populated, this advantage diminishes.     

Hybrid image‐based collision detection in Java 3D

Collision detection is highly important in computer graphics and virtual reality. Most collision detection methods are object‐based, relying on testing the geometrical interference of objects, and their performance therefore depends on the geometrical complexity of the objects. Recently, image‐based methods have gained increasing acceptance for their simplicity in implementation, robustness with respect to the object geometry, and the potential to distribute the computational burden onto graphics hardware. However, all existing image‐based methods require direct calls to OpenGL, but so far there is no direct way to access OpenGL through the Java 3D API. Although Java 3D provides its own built‐in collision detection classes, they are either incorrect or inefficient. In this paper, we present a hybrid image‐based collision detection method in Java 3D, which incorporates the Java 3D built‐in collision detection and the image‐based collision detection in our specially devised scene graph. In addition, we take advantage of the fact that the 3D position of successive offscreen views (i.e. virtual views perceived by the probing object) does not change significantly and thereby reduce the occurrences of offscreen rendering, so that the collision detection becomes even faster (up to 50% in our case). Experimental results prove the correctness and efficiency of our method. Copyright © 2006 John Wiley & Sons, Ltd.     

航空遥感图像深度学习目标检测技术研究进展

航空遥感图像目标检测旨在定位和识别遥感图像中感兴趣的目标,是航空遥感图像智能解译的关键技术,在情报侦察、灾害救援和资源勘探等领域具有重要应用价值。然而由于航空遥感图像具有尺寸大、目标小且密集、目标呈任意角度分布、目标易被遮挡、目标类别不均衡以及背景复杂等诸多特点,航空遥感图像目标检测目前仍然是极具挑战的任务。基于深度卷积神经网络的航空遥感图像目标检测方法因具有精度高、处理速度快等优点,受到了越来越多的关注。为推进基于深度学习的航空遥感图像目标检测技术的发展,本文对当前主流遥感图像目标检测方法,特别是2020—2022年提出的检测方法,进行了系统梳理和总结。首先梳理了基于深度学习目标检测方法的研究发展演化过程,然后对基于卷积神经网络和基于Transformer目标检测方法中的代表性算法进行分析总结,再后针对不同遥感图象应用场景的改进方法思路进行归纳,分析了典型算法的思路和特点,介绍了现有的公开航空遥感图像目标检测数据集,给出了典型算法的实验比较结果,最后给出现阶段航空遥感图像目标检测研究中所存在的问题,并对未来研究及发展趋势进行了展望。     

Designing large‐scale interactive traffic animations for urban modeling

Designing and optimizing traffic behavior and animation is a challenging problem of interest to virtual environment content generation and to urban planning and design. While some traffic simulation methods have appeared in computer graphics, most related systems focus on the design of buildings, roads, or cities but without explicitly considering urban traffic. To our knowledge, our work provides the first interactive approach which enables a designer to specify a desired vehicular traffic behavior (e.g., road occupancy, travel time, emissions, etc.) and the system will automatically compute what realistic 3D urban model (e.g., an interconnected network of roads, parcels, and buildings) yields the specified behavior. Our system both altered and improved traffic behavior in novel procedurally‐generated cities and in road networks of existing cities. Our urban models contain up to 360 km of roads, 300,000 vehicles, and typically cover four hours of simulated peak traffic time. The typical editing session time to “paint” a new traffic pattern and to compute the new/changed urban model is two to five minutes.     

Chiron parallel program performance visualization system

Chiron is a prototype visualization system for displaying the memory system performance of shared memory multiprocessor applications. The system uses 3D graphics techniques to display large amounts of both code-oriented and data-oriented information. Chiron is designed to isolate problems such as low cache block utilization, improper layout of data in memory resulting in excessive replacement interference, and improper partitioning of work among the processors resulting in excessive coherence interference. A 3D interactive user interface provides the user with flexibility in displaying the data and facilitates the job of focusing in on memory bottlenecks. The paper describes the design and implementation of Chiron, and illustrates its use.