A unified framework for document restoration using inpainting and shape-from-shading

We present a restoration framework to reduce undesirable distortions in imaged documents. Our framework is based on two components: (1) an image inpainting procedure that can separate non-uniform illumination (and other) artifacts from the printed content and (2) a shape-from-shading (SfS) formulation that can reconstruct the 3D shape of the document's surface. Used either piecewise or in its entirety, this framework can correct a variety of distortions including shading, shadow, ink-bleed, show-through, perspective and geometric distortions, for both camera-imaged and flatbed-imaged documents. Our overall framework is described in detail. In addition, our SfS formulation can be easily modified to target various illumination conditions to suit different real-world applications. Results on images of synthetic and real documents demonstrate the effectiveness of our approach. OCR results are also used to gauge the performance of our approach.     

Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth,Illumination and Albedo

Shape from shading (SfS) and stereo are two fundamentally different strategies for image-based 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on a matching process that establishes correspondences across images. This difference in approaching the reconstruction problem yields complementary advantages that are worthwhile being combined. So far, however, most “joint” approaches are based on an initial stereo mesh that is subsequently refined using shading information. In this paper we follow a completely different approach. We propose a joint variational method that combines both cues within a single minimisation framework. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detail-preserving anisotropic second-order smoothness term. Moreover, we extend the resulting model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes the approach applicable to objects with non-uniform albedo as well as to scenes with unknown illumination. Experiments for synthetic and real-world images demonstrate the benefits of our combined approach: They not only show that our method is capable of generating very detailed reconstructions, but also that joint approaches are feasible in practice.     

Efficiently determining a locally exact shortest path on polyhedral surfaces

In this paper, we present an efficient visibility-based algorithm for determining a locally exact shortest path (LESP) from a source point to a destination point on a (triangulated) polyhedral surface. Our algorithm, of a finitely-iterative scheme, evolves an initial approximately shortest path into a LESP. During each iteration, we first compute the exact shortest path restricted on the current face sequence according to Fermat’s principle which affirms that light always follows the shortest optical path, and then optimize the face sequence where the path is not locally shortest on the polyhedral surface. Since the series of paths we obtained are monotonic decreasing in length, the algorithm gives a LESP which is shorter than the initial path, at conclusion.

For comparison, we use various methods to provide an initial path. One of the methods is Dijkstra’s algorithm, and the others are the Fast Marching Method (FMM) and its improved version. Our intention for improvement is to overcome the limitation of acute triangulations in the original version. To achieve this goal, we classify all the edges into seven types according to different wavefront propagation manners, and dynamically determine the type of each edge for controlling the subsequent wavefront expansion. Furthermore, we give two approaches for backtracing the approximately shortest paths directed at the improved FMM. One exploits the known propagation manners of the edges as well as the Euler’s method. This is another contribution in this paper.     

Shape-from-Shading Under Perspective Projection

Shape-from-Shading (SfS) is a fundamental problem in Computer Vision. A very common assumption in this field is that image projection is orthographic. This paper re-examines the basis of SfS, the image irradiance equation, under a perspective projection assumption. The resultant equation does not depend on the depth function directly, but rather, on its natural logarithm. As such, it is invariant to scale changes of the depth function. A reconstruction method based on the perspective formula is then suggested; it is a modification of the Fast Marching method of Kimmel and Sethian. Following that, a comparison of the orthographic Fast Marching, perspective Fast Marching and the perspective algorithm of Prados and Faugeras on synthetic images is presented. The two perspective methods show better reconstruction results than the orthographic. The algorithm of Prados and Faugeras equates with the perspective Fast Marching. Following that, a comparison of the orthographic and perspective versions of the Fast Marching method on endoscopic images is introduced. The perspective algorithm outperformed the orthographic one. These findings suggest that the more realistic set of assumptions of perspective SfS improves reconstruction significantly with respect to orthographic SfS. The findings also provide evidence that perspective SfS can be used for real-life applications in fields such as endoscopy.This research has been supported in part by Tel-Aviv University fund, the Adams Super-Center for Brain Studies, the Israeli Ministry of Science, the ISF Center for Excellence in Applied Geometry, the Minerva Center for geometry, and the A.M.N. fund.     

Shape from silhouette using Dempster-Shafer theory

This work proposes a novel shape from silhouette (SfS) algorithm using the Dempster-Shafer (DS) theory for dealing with inconsistent silhouettes. Standard SfS methods makes assumptions about consistency in the silhouettes employed. However, total consistency hardly ever happens in realistic scenarios because of inaccuracies in the background subtraction or occlusions, thus leading to poor reconstruction outside of controlled environments.Our method classify voxels using the DS theory instead of the traditional intersection of all visual cones. Sensors reliability is modelled taking into account the positional relationships between camera pairs and voxels. This information is employed to determine the degree in which a voxel belongs to a foreground object. Finally, evidences collected from all sensors are fused to choose the best hypothesis that determines the voxel state.Experiments performed with synthetic and real data show that our proposal outperforms the traditional SfS method and other techniques specifically designed to deal with inconsistencies. In addition, our method includes a parameter for adjusting the precision of the reconstructions so that it could be adapted to the application requirements.     

Foreground prediction for bilayer segmentation of videos

Given a segmentation result (an alpha matte or a binary mask) of the former frame, foreground prediction is a process of estimating the probability that each pixel in the current frame belongs to the foreground. It plays a very important role in bilayer segmentation of videos, especially videos with non-static backgrounds. In this paper, a new foreground prediction algorithm which is called opacity propagation is proposed. It can propagate the opacity values of the former frame to the current frame by minimizing a cost function that is constructed by assuming the spatiotemporally local color smoothness of the video. Optical flow and probability density estimation based on a local color model are employed to find the corresponding pixels of two adjacent frames. An OPSIC (opacity propagation with sudden illumination changes) algorithm is also proposed which is an improvement of our proposed opacity propagation algorithm because it adds a simple color transformation model. As far as we know, this is the first algorithm that can predict the foreground accurately when the illumination changes suddenly. The opacity map (OM) generated by the opacity propagation algorithm is usually more accurate than the previously used probability map (PM). The experiments demonstrate the effectiveness of our algorithm.     

Shape from silhouettes based on a centripetal pentahedron model

In this paper we present a novel volumetric shape from silhouette (SfS) algorithm based on a centripetal pentahedron model (pent-model). The pent-model is an object-centered volumetric model composed of a set of pentahedrons cut from the centripetal triangular pyramids, which together partition the 3D space. The SfS algorithm first computes the pyramids by constructing a geodesic sphere. These pyramids are then projected onto the image planes of all cameras. The intersections between the projected pyramids and the silhouettes, which are a set of hexagons, are computed. This process can be performed very efficiently with pre-computed polar silhouette graphs (PSGs) and reduced PSGs. The hexagons are then back-projected into the 3D space, where the intersections are calculated and the pent-model is derived. After that, a mesh surface model can be extracted by marching pentahedrons. Our algorithm has the combined advantages of robustness, speediness and preciseness. Experimental results based on both synthetic images and real photos are presented.     

基于匹配扩散的多视稠密深度图估计

提出一种高精度的基于匹配扩散的稠密深度图估计算法. 算法分为像素级与区域级两阶段的匹配扩散过程.前者主要对视图间的稀疏特征点匹配进行扩散以获取相对稠密的初始深度图; 而后者则在多幅初始深度图的基础上, 根据场景分段平滑的假设, 在能量函数最小化框架下利用平面拟合及多方向平面扫描等方法解决存在匹配多义性问题区域(如弱纹理区域)的深度推断问题. 在标准数据集及真实数据集上的实验表明, 本文算法对视图中的光照变化、透视畸变等因素具有较强的适应性, 并能有效地对弱纹理区域的深度信息进行推断, 从而可以获得高精度、稠密的深度图.     

A modified fuzzy min–max neural network with rule extraction and its application to fault detection and classification

The fuzzy min–max (FMM) network is a supervised neural network classifier that forms hyperboxes for classification and prediction. In this paper, we propose modifications to FMM in an attempt to improve its classification performance when a small number of large hyperboxes are formed in the network. Given a new input pattern, in addition to measuring the fuzzy membership function of the input pattern to the hyperboxes formed in FMM, an Euclidean distance measure is introduced for predicting the target class associated with the new input pattern. A rule extraction algorithm is also embedded into the modified FMM network. A confidence factor is calculated for each FMM hyperbox, and a user-defined threshold is used to prune the hyperboxes with low confidence factors. Fuzzy if–then rules are then extracted from the pruned network. The benefits of the proposed modifications are twofold, viz., to improve the performance of FMM when large hyperboxes are formed in the network; to facilitate the extraction of a compact rule set from FMM to justify its predictions. To assess the effectiveness of modified FMM, two benchmark pattern classification problems are experimented, and the results from different methods published in the literature are compared. In addition, a fault detection and classification problem with a set of real sensor measurements collected from a power generation plant is evaluated using modified FMM. The results obtained are analyzed and explained, and implications of the modified FMM network as a useful fault detection and classification tool in real environments are discussed.     

A detection method for near-ship wakes based on interferometric magnitude,phase and physical shape in ATI-SAR systems

Near-ship wakes approximately form a triangle region on the sea surface, and they differ from the background sea in a wave pattern, wave propagation direction and velocity, and fluid particle orbital motion. The differences are observed by along-track interferometric synthetic aperture radar (ATI-SAR) systems, and can be properly measured by the ATI techniques. In this paper, a detection method for near-ship wakes based on interferometric magnitude, phase and physical shape in ATI-SAR systems is proposed. The complex multi-look interferometric information for near-ship wakes is first utilized to detect potential pixels. Then, a cluster algorithm is designed based on the interferometric phase and spatial position characteristics of near-ship wakes. After clustering the detected pixels and performing image close operation, the near-ship wakes, which is the triangular region with a much larger physical area than those of common ships or other false alarms, can be identified. Finally, simulation results validate the effectiveness of the proposed method.     

Analytical study of surface wave multiple refraction in boundary of a scalar impedance surface with a tensor impedance surface

In this paper, the multiple refraction phenomenon is investigated on the boundary of a scalar impedance surface (SIS) and a tensor impedance surface (TIS). When a surface wave (SW) propagates on the SIS and radiates to the boundary of the TIS, the propagation direction of it is changed and the refraction phenomenon is accrued. The method that is proposed in this paper can predict the multiple refraction for the SW. Moreover, another analytical method is introduced for designing the proposed structure which the double refraction (DR) occurs at arbitrary angles on it. Using it, a sample of the structure is designed by printed circuits in 15.2GHz and the results are verified by the full‐wave simulation and measurement. The results are shown that in the structure, DR is occurred in 2° and 22° as predicted. The proposed method can provide many applications such as design of SW power dividers based on the TISs, impedance surface based waveguides, holographic antennas, and feeding of array antennas.     

一种基于双层插值的路径规划及跟踪算法

针对基于栅格地图的经典路径规划算法的不足,提出了一种双层插值的路径规划算法——T* 算法． 算法首先应用快速行军插值法生成目标波形图,波形图假设波以目标点为中心经四个邻节点向外扩散且沿波传播方 向栅格效用值单调递增;然后在生成的目标波形图上采用线性插值法,从机器人当前位置沿着波逆向搜索一条到达 目标点的平滑路径;得到规划的路径后,针对非完整约束的移动机器人,采用改进的虚拟车辆方法跟踪生成的路径． 实验验证了算法的有效性．     

FMM算法在Cell/B.E.处理器上实现的分析与验证

FMM算法[1]是基于树结构的,用于解决多体问题(N-Body)的经典算法。它将N-Body问题的计算复杂度由O(N2)降为O(N),并且能达到任意精度。通用CPU在计算规模较大的N-Body问题时需要耗费大量的时间。为了加速算法的执行,本文对FMM算法在Cell/B.E.处理器上的实现进行了分析与验证。首先从功能上将FMM算法分解为八个核心过程,在此基础上根据计算特点的不同,对八个核心过程进行归类,最后选取其中有代表性的核心步骤,阐述了其在Cell/B.E.上实现的可行性问题,以及部分核心步骤的设计和实现过程。实验结果表明,选定的FMM算法核心步骤在Cell/B.E.上可以获得相对通用CPU较高的加速比。     

石英上准纵漏表面声波在液体传感方面的潜在应用

本文报道了石英材料上一种新的可用于液体传感的准纵漏声表面波模式的传播方向 ,即 Euler角为 [0°,136°,5 0°]的传播方向 ,并对该传播方向上准纵漏声表面波在液体传感方面的应用进行了初步研究 .结果表明 :与目前常用液体声表面波传感器所用声波模式所在传播方向相比较 ,这种新的传播方向上存在较少的声表面波模式 ,当应用于液体传感时只有准纵漏声表面波一种模式存在 .特别是 ,该传播方向上准纵漏声表面波频率温度系数实验测量值为 - 31.13ppm/℃ .这些特性使得本文所提传播方向上准纵漏声表面波具有很好的应用前景 .     

Adaptive ILC for tracking non-repetitive reference trajectory of 2-D FMM under random boundary condition

Almost all of the existing research achievements in Iterative Learning Control (ILC) hitherto have been focused on One-Dimensional (1-D) dynamical systems. Few ILC researches are related to Two-Dimensional Fornasini Marchesina Model (2-D FMM). In this paper, an adaptive ILC approach is proposed for 2-D FMM system with non-repetitive reference trajectory under random boundary condition. The proposed adaptive ILC algorithm learns the coefficient matrices of the system and updates the control input iteratively. As the times of iteration goes to infinity, the ILC tracking error outside the boundary tends to zero and all system signals keep bounded in the whole ILC process. Illustrative examples are provided to verify the validity of the proposed adaptive ILC algorithm.     

基于高动态范围图像中光晕分析的光照方向测算算法

针对增强现实(AR)系统中复杂场景的光照一致性问题,运用高动态范围(HDR)图像处理技术对标识物图像进行分析,提出一种基于HDR图像中光晕分析的光照方向测算算法。为了提高虚拟物体的沉浸感和真实性,在研究和分析现有的光照恢复算法的基础上,利用二次曲线对的投影不变性原理进行相机标定;为了获得更加详尽的光照信息,使用HDR技术对标识物图像进行处理,提高了测算精确度;参照Lambert光照模型对图像中的光照信息进行分析,将拍摄角度进行分类,实现了对传统测算光源方向算法的改进,可测算出位于摄影球反射范围外的部分光源的方向,扩大了测算范围。设计了针对单一点光源的视角1和视角2测算验证实验,并进行了分析。实验验证表明,所提方法简单易行,具有较强的鲁棒性,能够实现在标识物部分遮挡的情况下,测算出位于摄影球反射范围外部分光源的方向。     

Guided depth enhancement via a fast marching method

Range imaging sensors such as Kinect and time-of-flight cameras can produce aligned depth and color images in real time. However, the depth maps captured by such sensors contain numerous invalid regions and suffer from heavy noise. These defects more or less influence the use of depth information in practical applications. In order to enhance the depth maps, this paper proposes a new inpainting approach based on the fast marching method (FMM). We extend the inpainting model and the propagation strategy of FMM to incorporate color information for depth inpainting. An edge-preserving guided filter is further applied for noise reduction. To validate our algorithm, we perform experiments on both Kinect data and Middlebury dataset which, respectively, provide qualitative and quantitative results. Meanwhile, we also compare it to the original FMM and other two state-of-the-art depth enhancement methods. Experimental results show that our method performs better than the local methods in terms of both visual and metric qualities, and it achieves visually comparable results to the time-consuming global method.     

基于多聚焦图像融合的小孔内表面缺陷检测

针对传统直径3 mm以下小孔内表面缺陷检测缺乏有效检测方法的问题,提出了一种新的基于显微光学与多聚焦图像融合的小孔内表面缺陷检测方法。首先在不同光照环境下,呈斜入射方式沿小孔轴线方向依次采集单侧孔壁的序列图像;然后通过前景光照下得到的掩膜模板提取孔壁图像的感兴趣区域（ROI）,并运用快速鲁棒性特征（SURF）算法实现ROI配准;进而采用基于区域清晰度的小波图像融合方法实现ROI多聚焦图像融合;最后以直径2 mm喷丝板导孔为实验对象,采用阈值分割算法提取导孔内表面的溶蚀斑进行检测分析。实验结果表明,该方法具有一定的可行性,避免了传统人工检测方法效率低的问题,同时打破了传统直径3 mm以下的小孔内表面检测方法不再适用的局限性。     

自适应最小误差阈值分割算法

对二维最小误差法进行三维推广, 并结合三维直方图重建和降维思想提出了一种鲁 棒的最小误差阈值分割算法. 但该方法为全局算法, 仅适用于分割均匀光照图像. 为 提高其自适应性, 本文采用Water flow模型对非均匀光照图像进行背景估计, 以此获 得原始图像与背景图像的差值图像, 达到降低非均匀光照对图像分割造成干扰的目的. 为进 一步提高分割性能, 本文对差值图像采用γ 矫正进行增强, 然后采用鲁棒最小误差 法进行全局分割, 从而完成目标提取. 最后本文对均匀光照下以及非均匀光照下图像进行了 实验, 并与一维最小误差法、二维最小误差法、三维直方图重建和降维的Otsu阈值分割 算法、灰度波动变换自适应阈值方法以及一种改进的FCM方法在错误分割率和运行时间上进 行了对比. 实验结果表明, 相对于以上方法, 本算法的分割性能均有明显提升.     

Fast multipole method based solution of electrostatic and magnetostatic field problems

Abstact Applications of boundary element methods (BEM) to the solution of static field problems in electrical engineering are considered in this paper. The choice of a suitable BEM formulation for electrostatics, steady current flow fields or magnetostatics is discussed from user's point of view. The dense BEM matrix is compressed with an enhanced fast multipole method (FMM) which combines well-known BEM techniques with the FMM approach. An adaptive grouping scheme for problem oriented meshes is presented along with a discussion on the influence of the mesh to the efficiency of the FMM. The computational costs of the FMM algorithm are analyzed for typical problems in practice. Finally, some electrostatic and magnetostatic numerical examples demonstrate the simple usability and the efficiency of the FMM. Communicated by: U. Langer