开源跨平台的图像可视化调试器设计

目的 在调试C/C++图像处理程序时,如何以可视化的形式观察被调试程序中的图像变量,对于调试过程尤为关键。目前尚未有跨多操作系统平台的图像可视化调试器可供使用,该款开源跨平台的图像可视化调试器的设计与实现弥补了此领域的不足。方法 利用GDB(GNU debugger)调试器的Python接口,将被调试程序中的图像变量对应的内存字节序列转化成Python下的2维数组,并采用Matplotlib库加以显示,整个图像显示线程独立于GDB的文本字符交互主线程。结果 在Windows、Linux、Mac系统中分别进行实验,均可实现图像变量的显示、缩放、平移、像素数值查看、保存等多种功能,并使得GDB的命令行保持非阻塞模式运行。结论 开源跨平台的图像可视化调试器的设计,满足了不同操作系统平台下图像程序的开发调试需求,弥补了当前GDB调试图像程序功能的不足,提高了图像处理程序的开发和调试效率。     

面向智慧城市的3维城市在线可视化

目的 3维城市可视化是智慧城市信息显示的基础,对城市信息的实时准确传递起着重要作用。而现有的3维城市可视化方法和系统存在两点局限性:一是数据模型不适合于海量建筑物显示;二是对整个城市采用单一绘制方式,而建筑物的纹理、结构、高度等特征相似,绘制结果容易引起视觉混淆,为此提出一种基于人类感知理论的3维城市在线可视化技术。方法在预处理阶段,系统采用建筑综合算法建立3维城市建筑物的多分辨率表示;在运行时刻,系统根据用户交互,自适应选择建筑物相应的层次进行显示。结果采用几个3维城市数据对系统进行了测试,实验结果证明,该系统有效地提高了3维城市绘制效率。Leverkusen城市的5 530座建筑物,绘制效率达到19.4帧/s。结论基于感知的3维城市多分辨率表示,有效提高了3维城市系统的显示效率以及用户获取信息的效率,同时提高了用户的交互效率。     

基于超像素的多主体图像交互分割

目的 为解决多主体图像的交互分割问题,在保证分割效果的前提上,提高分割的效率,达到实时交互修改分割结果的目的, 提出基于超像素的图像多主体交互分割算法.方法 基于图像的超像素构造一个多层流网络,利用用户交互绘制的简单笔画给出多主体分割的指导信息.流网络的边权值保证利用图割算法将图像分割成多个部分后,每个部分代表图像的一个主体.允许用户交互给出标记,实时修改分割结果,直到得到满意的多主体分割.结果 通过实验显示,本文方法能得到的满意多主体分割结果,而且时间效率较高.对分辨率为449×275的图像,算法能在1 s内给出结果,满足实时修改的要求.结论 基于超像素建立的图规模较小,能大大减少图割算法的运行时间,达到用户实时交互添加新笔画信息,交互地修正分割结果的目的.利用超像素的边界信息,用户只需输入比较简单的笔画信息,分割算法就能得到正确的多主体分割结果.     

基于WebVR的医学体数据可视化

传统的医学体数据可视化通过计算机屏幕显示三维医学影像,由于使用者对二维屏幕缺乏深度感知,对于组织内各部位无法准确判断相对位置,难以观察理解。为了解决这个问题,结合虚拟现实(VR)技术与体绘制技术,提出并实现基于WebVR的医学体数据可视化系统。基于B/S模式设计系统的整体架构,浏览器端从服务器端获得体数据,使用WebGL调用本地显卡对渲染进行硬件加速。运用光线投射算法进行体绘制,渲染结果通过浏览器显示或使用WebVR连接至VR设备显示在虚拟空间中,实现沉浸式的可视化。同时,针对虚拟现实环境对光线投射算法提出优化方法,实现激光拾取、体数据内部可视化和平面裁剪的交互方式。系统实现了新的三维立体视觉的感知方式和方便灵活的交互功能,增强使用者对三维医学体数据可视化影像的感知。     

图形图像融合的海量建筑绘制

目的 城市3维模型数据海量且结构复杂,缺乏一个高效完善的可视化系统往往是影响数字城市应用的瓶颈之一。通常利用多层次细节（LOD）与调度算法减少每一帧绘制的数据量来提高绘制效率,当场景规模足够大时,即使采用复杂的优化算法也难以取得较好的效果。为此,本文在传统算法基础上,提出一种图形图像融合的海量建筑物场景绘制方法。方法 提出并采用视域分级绘制策略,将视椎体平行分割为感兴趣区域、次感兴趣区域和非感兴趣区域,感兴趣区域采用图形实时绘制方法,使用离屏渲染技术将次感兴趣和非感兴趣区域绘制在纹理图像中,每一帧绘制完成后将二者进行顾及深度信息的融合,实现完整场景渲染。结果 使用公开的纽约市区CityGML文件作为实验数据,数据包含了118 195个LOD1和LOD2级别的建筑物模型。分别构建多组不同建筑数量的场景进行帧率统计实验,绘制帧率都达到20帧/s以上。算法实现了视觉无损失的场景完整渲染,并与Cesium平台进行对比实验,证明算法有效且系统运行流畅。结论 图形图像融合的绘制方法,既保持了图形渲染的漫游连续性,同时也具有图像渲染的场景复杂度无关的优点。实验结果表明,针对大规模的低分辨率建筑模型场景,算法可以有效提高系统的渲染能力,在性能相对较低的硬件条件下也能实现海量建筑物实体模型的流畅漫游,并达到视觉无损失的场景完整绘制。     

应用于人体关节缺损修复的建模与可视化

目的 关节缺损疾病治疗目前存在的主要问题是缺乏精确的关节模型以及个体化的修复方案,为此提出量化关节骨缺损的精确建模与可视化方法。方法 利用骨骼图像增强、多模态影像融合、关节结构分割、关节病变结构建模与定量分析等核心关键技术,从关节CT或MRI影像中构建和恢复缺损关节的空间立体结构,为关节缺损信息提供量化参数和3维模型,从而帮助医生快速准确地对关节缺损疾病进行诊疗。结果 针对建模与可视化方法中的核心关键技术进行了深入研究,实验结果显示上述方法能够为关节缺损修复提供精确的3维量化模型。结论 基于CT、MRI影像的关节结构建模与可视化技术为评价骨缺损大小提供了精确有效的方法,在关节盂或肱骨结构等疾病的诊疗方面具有重要的临床意义,此项技术的发展对关节缺损疾病的修复发挥着重要作用。     

一种全球尺度三维大气数据可视化系统

气象数据具有时变、多源、多维度、数据量大和多尺度等特征,常规的气象数据可视化方法难以满足气象预报与气象研究中的需求.描述了一个新的全球尺度三维大气数据可视化系统——AVIS.系统实现了面向各类密度场、向量场、张量场和非空间数据的可视化方法,设计了适用于大气数据的球面体绘制和混合绘制方法.系统还实现了一套跨平台并行可视化及分析构架,支持在浏览器中或其他显示和交互设备上使用,并利用后端计算集群加速数据的计算与绘制过程.案例分析表明,该系统可以全方位地展现气象数据中的信息,帮助用户综合多种气象信息进行分析.     

肺部影像解剖结构分割数据集及应用

目的 从影像中快速精准地分割出肺部解剖结构可以清晰直观地分辨各解剖结构间的关系,提供有效、客观的辅助诊断信息,大大提高医生的阅片效率并降低医生的工作量。随着影像分割算法的发展,越来越多的方法应用于分割肺部影像中感兴趣的解剖结构区域,但目前尚缺乏包含多种肺部精细解剖结构的影像数据集。本文创建了一个带标签的肺部CT/CTA （computer tomography/computer tomography angiography）影像数据集,以促进肺部解剖结构分割算法的发展。方法 该数据集共标记了67组肺部CT/CTA影像,包括CT影像24组、CTA影像43组,共计切片图像26 157幅。每组CT/CTA有4个不同的目标区域类别,标记对应支气管、肺实质、肺叶、肺动脉和肺静脉。结果 本文利用该数据集,用于肺部CT解剖结构分割医学影像挑战赛——2020年第四届国际图像计算与数字医学研讨会,该挑战赛提供了一个肺血管、支气管和肺实质的评估平台,通过Dice系数、过分割率、欠分割率、医学和算法行业专家对分割和3维重建效果进行了评估,目的是比较各种算法分割肺部解剖结构的性能。结论 本文详细描述了包括支气管、肺实质、肺叶、肺动脉和肺静脉等解剖结构标签的肺部影像数据集和应用结果,为相关研究人员利用本数据集进行更深入的研究提供参考。     

结合背景图的高分辨率视频人像实时抠图网络

目的 近年来,采用神经网络完成人像实时抠图已成为计算机视觉领域的研究热点,现有相关网络在处理高分辨率视频时还无法满足实时性要求,为此本文提出一种结合背景图的高分辨率视频人像实时抠图网络。方法 给出一种由基准网络和精细化网络构成的双层网络,在基准网络中,视频帧通过编码器模块提取图像的多尺度特征,采用金字塔池化模块融合这些特征作为循环解码器网络的输入;在循环解码器中,通过残差门控循环单元聚合连续视频帧间的时间信息,以此生成蒙版图、前景残差图和隐藏特征图,采用残差结构降低模型参数量并提高网络的实时性。为提高高分辨率图像实时抠图性能,在精细化网络中,设计高分辨率信息指导模块,通过高分辨率图像信息指导低分辨率图像的方式生成高质量人像抠图结果。结果 与近年来的相关网络模型进行实验对比,实验结果表明,本文方法在高分辨率数据集Human2K上优于现有相关方法,在评价指标(绝对误差、均方误差、梯度、连通性)上分别提升了18.8%、39.2%、40.7%、20.9%。在NVIDIA GTX 1080Ti GPU上处理4 K分辨率影像运行速率可达26帧/s,处理HD(high definition)分辨率影像运行速率可达43帧/s。结论 本文模型能够更好地完成高分辨率人像实时抠图任务,可以为影视、短视频社交以及网络会议等高级应用提供更好的支持。     

非对称高斯函数的时变体数据特征跟踪及可视化

目的 在传统基于时序曲线的时变体数据特征的识别与跟踪过程中,不仅需要用户具备丰富的先验知识来确定感兴趣特征的时序曲线形状,而且时序曲线段的匹配和抽取过程亦复杂,难以交互,这一定程度上降低了时变体数据可视化及分析效率.提出了一种新的基于高斯时序曲线的时变体数据可视化及分析方法.方法 首先,对时序曲线进行低通滤波,准确地查找极小值点,对时序曲线进行分段;进而,引入非对称高斯函数对时序曲线段进行拟合获得高斯时序曲线;为了进一步方便用户识别感兴趣特征,观察特征在时间域上的演变过程,提供一种便捷的交互技术,由用户单击任一时间步绘制结果图像中感兴趣的特征信息,分析视线方向上的特征可见性,以确定感兴趣特征及其对应的高斯曲线,进而由参数匹配获得所有时间步中感兴趣特征信息.结果 为了验证本文算法的高效性与可行性,对时序仿真的飓风数据进行特征跟踪与交互式可视化,可以看出,本文算法不仅可以准确跟踪飓风中心特征,而且特征分析与可视化效率亦大大提升.结论 相比于传统的时变体数据可视化方法,本文算法不需要用户先验知识的前提下,利用非对称高斯模拟时序曲线变化,进而由高斯参数匹配代替复杂的时序曲线匹配过程,有效地提升了时变体数据可视化及分析效率.     

Interactive, internet delivery of visualization via structured prerendered multiresolution imagery

We present a novel approach for latency-tolerant delivery of visualization and rendering results where client-side frame rate display performance is independent of source dataset size, image size, visualization technique or rendering complexity. Our approach delivers pre-rendered, multiresolution images to a remote user as they navigate through different viewpoints, visualization or rendering parameters. We employ demand-driven tiled, multiresolution image streaming and prefetching to efficiently utilize available bandwidth while providing the maximum resolution user can perceive from a given viewpoint. Since image data is the only input to our system, our approach is generally applicable to all visualization and graphics rendering applications capable of generating image files in an ordered fashion. In our implementation, a normal web server provides on-demand images to a remote custom client application, which uses client-pull to obtain and cache only those images required to fulfill the interaction needs. The main contributions of this work are: (1) an architecture for latency-tolerant, remote delivery of precomputed imagery suitable for use with any visualization or rendering application capable of producing images in an ordered fashion; (2) a performance study showing the impact of diverse network environments and different tunable system parameters on end-to-end system performance in terms of deliverable frames per second.     

渲染器与Web服务器耦合实现远程体 渲染的交互优化

目的 为使用户随时随地获得需要较大计算和存储资源的交互真实感体渲染服务,设计并实现了一种面向Web的远程真实感体渲染方法。方法 计算量较大的实时体渲染任务由远端渲染服务器中的GPU加速完成并通过WebSocket将渲染图像发送至客户端;客户端浏览器只需负责接收显示渲染图像并监听发送用户交互事件。提出了一种输出系统耦合算法用以连接输出图像速率较大的渲染服务器和发送图像速率较慢的Web服务器。算法能根据Web服务器发送图像的情况动态调整每次图像输出的迭代计算次数,改变渲染服务器输出图像的时间间隔以达到与Web服务器发送速度相平衡,同时保持渲染服务器持续工作。结果 实验比较了在不同网络传输条件下,采用输出系统耦合算法与直接连接渲染器和Web服务器,渲染4个不同数据集所需的完成时间及帧率等性能评价指标。在局域网和广域网环境下,本文方法分别最多只需17 s和14 s即可完成整个渲染过程,而采用直接连接渲染器和Web服务器的方法则分别至少需要31 s和60 s才能完成整个渲染过程。实验结果表明采用输出系统耦合算法在不同网络条件下均可较大地缩短整个渲染过程所需时间,使用户在较短时间内获得高质量渲染图像。结论 本文渲染器与Web服务器耦合实现远程体渲染交互优化的方法可让用户使用与计算能力无关的桌面或移动设备通过网络透明使用高性能渲染系统;系统采用的输出系统耦合算法能够根据网络承载能力自适应调整渲染器输出速度,使用户在不同的网络环境中均可以较快的速度获得高质量渲染图像。     

Web-Based Remote Renderingwith IBRAC (Image-Based Rendering Acceleration and Compression)

Recent advances in Internet and computer graphics stimulate intensive use and development of 3D graphics on the World Wide Web. To increase efficiency of systems using 3D graphics on the web, the presented method utilizes previously rendered and transmitted images to accelerate the rendering and compression of new synthetic scene images. The algorithm employs ray casting and epipolar constraints to exploit spatial and temporal coherence between the current and previously rendered images. The reprojection of color and visibility data accelerates the computation of new images. The rendering method intrinsically computes a residual image, based on a user specified error tolerance that balances image quality against computation time and bandwidth. Encoding and decoding uses the same algorithm, so the transmitted residual image consists only of significant data without addresses or offsets. We measure rendering speed-ups of four to seven without visible degradation. Compression ratios per frame are a factor of two to ten better than MPEG2 in our test cases. There is no transmission of 3D scene data to delay the first image. The efficiency of the server and client generally increases with scene complexity or data size since the rendering time is predominantly a function of image size. This approach is attractive for remote rendering applications such as web-based scientific visualization where a client system may be a relatively low-performance machine and limited network bandwidth makes transmission of large 3D data impractical.     

SAGE2: A collaboration portal for scalable resolution displays

In this paper, we present SAGE2, a software framework that enables local and remote collaboration on Scalable Resolution Display Environments (SRDE). An SRDE can be any configuration of displays, ranging from a single monitor to a wall of tiled flat-panel displays. SAGE2 creates a seamless ultra-high resolution desktop across the SRDE. Users can wirelessly connect to the SRDE with their own devices in order to interact with the system. Many users can simultaneously utilize a drag-and-drop interface to transfer local documents and show them on the SRDE, use a mouse pointer and keyboard to interact with existing content that is on the SRDE and share their screen so that it is viewable to all. SAGE2 can be used in many configurations and is able to support many communities working with various types of media and high-resolution content, from research meetings to creative session to education.SAGE2 is browser-based, utilizing a web server to host content, WebSockets for message passing and HTML with JavaScript for rendering and interaction. Recent web developments, with the emergence of HTML5, have allowed browsers to use advanced rendering techniques without requiring plug-ins (canvas drawing, WebGL 3D rendering, native video player, etc.). One major benefit of browser-based software is that there are no installation requirements for users and it is inherently cross-platform. A user simply needs a web browser on the device he/she wishes to use as an interaction tool for the SRDE. This lowers considerably the barrier of entry to engage in meaningful collaboration sessions.     

Basic support for cooperative work on the World Wide Web

The emergence and widespread adoption of the World Wide Web offers a great deal of potential in supporting cross-platform cooperative work within widely dispersed working groups. The Basic Support for Cooperative Work (BSCW) project at GMD is attempting to realize this potential through development of web-based tools which provide cross-platform collaboration services to groups using existing web technologies. This paper describes one of these tools, theBSCW Shared Workspace system—a centralized cooperative application integrated with an unmodified web server and accessible from standard web browsers. The BSCW system supports cooperation through “shared workspaces”; small repositories in which users can upload documents, hold threaded discussions and obtain information on the previous activities of other users to coordinate their own work. The current version of the system is described in detail, including design choices resulting from use of the web as a cooperation platform and feedback from users following the release of a previous version of BSCW to the public domain.     

Web video thumbnail recommendation with content-aware analysis and query-sensitive matching

In this paper, a unified and adaptive web video thumbnail recommendation framework is proposed, which recommends thumbnails both for video owners and browsers on the basis of image quality assessment, image accessibility analysis, video content representativeness analysis and query-sensitive matching. At the very start, video shot detection is performed and the highest image quality video frame is extracted as the key frame for each shot on the basis of our proposed image quality assessment method. These key frames are utilized as the thumbnail candidates for the following processes. In the image quality assessment, the normalized variance autofocusing function is employed to evaluate the image blur and ensures that the selected video thumbnail candidates are clear and have high image quality. For accessibility analysis, color moment, visual salience and texture are used with a support vector regression model to predict the candidates’ accessibility score, which ensures that the recommended thumbnail’s ROIs are big enough and it is very accessible for users. For content representativeness analysis, the mutual reinforcement algorithm is adopted in the entire video to obtain the candidates’ representativeness score, which ensures that the final thumbnail is representative enough for users to catch the main video contents at a glance. Considering browsers’ query intent, a relevant model is designed to recommend more personalized thumbnails for certain browsers. Finally, by flexibly fusing the above analysis results, the final adaptive recommendation work is accomplished. Experimental results and subjective evaluations demonstrate the effectiveness of the proposed approach. Compared with the existing web video thumbnail generation methods, the thumbnails for video owners not only reflect the contents of the video better, but also make users feel more comfortable. The thumbnails for video browsers directly reflect their preference, which greatly enhances their user experience.     

适用于多模式虚拟学习的交互医学可视化及模拟

医学图像及其衍生的知识是医学教育中的一个关键学习部分，透过不同的图像模式，町以使医生及医科学生更有效地学习人体的体内结构．而计算机可视化技术最近的发展正满足了医生、外科医生及医科学生与日俱增的需求，为深入地了解人体生理学及病理学提供了宝贵的资料．本文提出了一个网上交互医学学习系统，并提供了一个虚拟现实工具包以支持二维医学图像序列观察、体数据和表面数据的三维可视化、目视放大镜．以及医学数据的交互变形模拟．其中人体组织物理特性的模拟以有限元法为基准．通过优化而获得有关的变形模型参数．该工具包可应用于个性化的二维及三维数据场可视化、优化的学习及手术规划和在线的交互医学教育．     

A unified framework for web video topic discovery and visualization

Together with the explosive growth of web video in sharing sites like YouTube, automatic topic discovery and visualization have become increasingly important in helping to organize and navigate such large-scale videos. Previous work dealt with the topic discovery and visualization problem separately, and did not take fully into account of the distinctive characteristics of multi-modality and sparsity in web video features. This paper tries to solve web video topic discovery problem with visualization under a single framework, and proposes a Star-structured K-partite Graph based co-clustering and ranking framework, which consists of three stages: (1) firstly, represent the web videos and their multi-model features (e.g., keyword, near-duplicate keyframe, near-duplicate aural frame, etc.) as a Star-structured K-partite Graph; (2) secondly, group videos and their features simultaneously into clusters (topics) and organize the generated clusters as a linked cluster network; (3) finally, rank each type of nodes in the linked cluster network by “popularity” and visualize them as a novel interface to let user interactively browse topics in multi-level scales. Experiments on a YouTube benchmark dataset demonstrate the flexibility and effectiveness of our proposed framework.     

WhiteScript: Using social network analysis parameters to balance between browser usability and malware exposure

Drive-by-download malware exposes internet users to infection of their personal computers, which can occur simply by visiting a website containing malicious content. This can lead to a major threat to the user’s most sensitive information. Popular browsers such as Firefox, Internet Explorer and Maxthon have extensions that block JavaScript, Flash and other executable content. Some extensions globally block all dynamic content, and in others the user needs to specifically enable the content for each site (s)he trusts. Since most of the web-pages today contain dynamic content, disabling them damages user experience and page usability, and that prevents many users from installing security extensions. We propose a novel approach, based on Social Network Analysis parameters, that predicts the user trust perspective for the HTML page currently being viewed. Our system examines the URL that appears in the address bar of the browser and each of the inner HTML URL reputations, and only if all of them have a reputation greater than our predetermined threshold, it marks the webpage as trusted. Each URL reputation is calculated based on the number and quality of the links on the whole web pointing back to the URL. The method was examined on a corpus of 44,429 malware domains and on the top 2000 most popular Alexa sites. Our system managed to enable dynamic content of 70% of the most popular websites and block 100% of malware web-pages, all without any user intervention. Our approach can augment most browser security applications and enhance their effectiveness, thus encouraging more users to install these important applications.