Estimating light vectors in real time

Capturing the appearance of objects under different lighting conditions is useful in texture acquisition, bidirectional reflectance distribution function (BRDF) measurement, and image-based data acquisition. With the acquired data, we can render virtual objects realistically. During the data capture, we must record both the radiance and the light vector. Measuring the light vector is not necessarily easy. We propose a capturing system that estimates the light vector by applying a vision-based technique. It lets the user freely position a handheld light source during capture. The system estimates light-source orientation in real time through a Web cam mounted on the light source. Software running on an ordinary PC analyzes images from the Web cam to recognize the light source direction (the light vector). Our goal is to design a low-cost, portable, and adaptable system. Vision-based approaches offer such characteristics. We use a pose-estimation system to acquire image-based data for relighting.     

Mixed reality based respiratory liver tumor puncture navigation

Computational Visual Media - This paper presents a novel mixed reality based navigation system for accurate respiratory liver tumor punctures in radiofrequency ablation (RFA). Our system contains...     

Discrete Wavelet Transform on Consumer-Level Graphics Hardware

Discrete wavelet transform (DWT) has been heavily studied and developed in various scientific and engineering fields. Its multiresolution and locality nature facilitates applications requiring progressiveness and capturing high-frequency details. However, when dealing with enormous data volume, its performance may drastically reduce. On the other hand, with the recent advances in consumer-level graphics hardware, personal computers nowadays usually equip with a graphics processing unit (GPU) based graphics accelerator which offers SIMD-based parallel processing power. This paper presents a SIMD algorithm that performs the convolution-based DWT completely on a GPU, which brings us significant performance gain on a normal PC without extra cost. Although the forward and inverse wavelet transforms are mathematically different, the proposed algorithm unifies them to an almost identical process that can be efficiently implemented on GPU. Different wavelet kernels and boundary extension schemes can be easily incorporated by simply modifying input parameters. To demonstrate its applicability and performance, we apply it to wavelet-based geometric design, stylized image processing, texture-illuminance decoupling, and JPEG2000 image encoding     

Shape matching and modeling using skeletal context

Shape is a significant visual clue for human perception and shape models show considerable promise as a basis for extracting objects from images. This paper proposes a novel approach for shape matching and modeling using the symmetry characterization of shape interior and the spatial relationships of shape structures. Based on the representative skeletal features, we develop a mechanism to generate a coarse segment matching between different instances of an object. Additionally, the natural correspondence of skeletal branches to sequential segments along the shape curves is employed in the matching process to avoid false correspondences across different segments. Point matches within the corresponding segments are then obtained by solving a constrained assignment problem. The validation of the proposed approach is illustrated on various data sets in the presence of considerable deformation and occlusion and the results are compared with those of popular approaches. We also demonstrate the performance of our method on biological objects for shape modeling, showing better models than those obtained by the state-of-the-art shape modeling approaches.     

Intelligent inferencing and haptic simulation for Chinese acupuncture learning and training.

This paper presents an intelligent virtual environment for Chinese acupuncture learning and training using state-of-the-art virtual reality technology. It is the first step toward developing a comprehensive virtual human model for studying Chinese medicine. Students can learn and practice acupuncture in the proposed 3-D interactive virtual environment that supports a force feedback interface for needle insertion. Thus, students not only "see" but also "touch" the virtual patient. With high performance computers, highly informative and flexible visualization of acupuncture points of various related meridian and collateral can be highlighted to guide the students during training. A computer-based expert system using our newly proposed intelligent fuzzy petri net is designed and implemented to train the students to treat different diseases using acupuncture. Such an intelligent virtual reality system can provide an interesting and effective learning environment for Chinese acupuncture.     

面向手术器械语义分割的半监督时空Transformer网络

基于内窥镜的微创手术机器人在临床上的应用日益广泛,为医生提供内窥镜视频中精准的手术器械分割信息,对提高医生操作的准确度、改善患者预后有重要意义.现阶段,深度学习框架训练手术器械分割模型需要大量精准标注的术中视频数据,然而视频数据标注成本较高,在一定程度上限制了深度学习在该任务上的应用.目前的半监督方法通过预测与插帧,可以改善稀疏标注视频的时序信息与数据多样性,从而在有限标注数据下提高分割精度,但是这些方法在插帧质量与对连续帧时序特征方面存在一定缺陷.针对此问题,提出了一种带有时空Transformer的半监督分割框架,该方法可以通过高精度插帧与生成伪标签来提高稀疏标注视频数据集的时序一致性与数据多样性,在分割网络bottleneck位置使用Transformer模块,并利用其自我注意力机制,从时间与空间两个角度分析全局上下文信息,增强高级语义特征,改善分割网络对复杂环境的感知能力,克服手术视频中各类干扰从而提高分割效果.提出的半监督时空Transformer网络在仅使用30%带标签数据的情况下,在MICCAI2017手术器械分割挑战赛数据集上取得了平均DICE为82.42%、平均IoU...     

GPU-friendly warped display for scope-maintained video surveillance

In this paper, we propose an efficient technique for warped display of surveillance video signal. Usually, there are regions of interest (ROIs) in video surveillance, such as entrance or exit, and moving objects or persons. The surveillant has two or more conflicting goals in mind. He/She wants to blow up the ROIs, but also wants to have an overview of the whole surveillance scope. The proposed method solves this conflict by warping the input video signal in real time fashion. First, multiple ROIs are detected. Then, the mesh for texture mapping is deformed, according to the position and shape of the ROIs. The original video frame is mapped as a texture on the deformed mesh to produce the warped effect. Mesh deformation and texture mapping are accelerated by modern graphics processing units (GPU). By allocating more display space for ROIs, the proposed technique highlights such important regions and assists the surveillant to locate them effectively. By experiments, we show that the proposed method is effective and efficient.     

Fuzzified Choquet Integral with a Fuzzy-valued Integrand and its application on temperature prediction.

In this paper, the original Choquet integral is generalized as a Fuzzified Choquet Integral with a Fuzzy-valued Integrand (FCIFI), which supports a fuzzy-valued integrand and an integration result. The calculation of the FCIFI is established on the Choquet integral with an interval-valued integrand (CIII). The definitions, properties, and calculation algorithms of the CIII and the FCIFI are discussed and proposed in this paper. As a specific application scheme, we designed a CIII regression model for the regression problems involving interval-valued data. This CIII regression model has a self-learning ability through a double genetic algorithm. Finally, a daily temperature predictor based on the CIII regression model is discussed, where a series of experiments is implemented to validate the performance of the predictor by real weather records from the Hong Kong Observatory.