Towards adaptive learning with improved convergence of deep belief networks on graphics processing units

In this paper we focus on two complementary approaches to significantly decrease pre-training time of a deep belief network (DBN). First, we propose an adaptive step size technique to enhance the convergence of the contrastive divergence (CD) algorithm, thereby reducing the number of epochs to train the restricted Boltzmann machine (RBM) that supports the DBN infrastructure. Second, we present a highly scalable graphics processing unit (GPU) parallel implementation of the CD-k algorithm, which boosts notably the training speed. Additionally, extensive experiments are conducted on the MNIST and the HHreco databases. The results suggest that the maximum useful depth of a DBN is related to the number and quality of the training samples. Moreover, it was found that the lower-level layer plays a fundamental role for building successful DBN models. Furthermore, the results contradict the pre-conceived idea that all the layers should be pre-trained. Finally, it is shown that by incorporating multiple back-propagation (MBP) layers, the DBNs generalization capability is remarkably improved.     

Improved GPU SIMD control flow efficiency via hybrid warp size mechanism

High single instruction multiple data (SIMD) efficiency and low power consumption have made graphic processing units (GPUs) an ideal platform for many complex computational applications. Thousands of threads can be created by programmers and grouped into fixed-size SIMD batches, known as warps. High throughput is then achieved by concurrently executing such warps with minimal control overhead. However, if a branch instruction occurs, which assigns different paths to different threads, one warp will be broken into multiple warps that have to be executed serially, consequently reducing the efficiency advantage of SIMD. In this paper, the contemporary fixed-size warp design is abandoned for a hybrid warp size (HWS) mechanism. Mixed-size warps are generated according to HWS and are scheduled and issued flexibly. The simulation results show that this mechanism yields an average speedup of 1.20 over the baseline architecture for a wide variety of general purpose GPU applications. The paper also integrates HWS with dynamic warp formation (DWF), which is a well-known branch handling mechanism used to improve SIMD utilization by forming new warps out of split warps in real time. The simulation results show that the combination of DWF and HWS generates an average speedup of 1.27 over the DWF-only platform with an estimated area increase of about 1% of DWF.     

Adaptive block size for dense QR factorization in hybrid CPU–GPU systems via statistical modeling

QR factorization is a computational kernel of scientific computing. How can the latest computer be used to accelerate this task? We investigate this topic by proposing a dense QR factorization algorithm with adaptive block sizes on a hybrid system that contains a central processing unit (CPU) and a graphic processing unit (GPU). To maximize the use of CPU and GPU, we develop an adaptive scheme that chooses block size at each iteration. The decision is based on statistical surrogate models of performance and an online monitor, which avoids unexpected occasional performance drops. We modify the highly optimized CPU–GPU based QR factorization in MAGMA to implement the proposed schemes. Numerical results suggest that our approaches are efficient and can lead to near-optimal block sizes. The proposed algorithm can be extended to other one-sided factorizations, such as LU and Cholesky factorizations.     

Tele‐Cocooning: Mobile Texting and Social Scope

The present study examines the tele‐cocooning hypothesis in the context of general trust using a nationally representative survey of Japanese youth. We find that although frequency of texting is positively correlated with general trust, this correlation is spuriously caused by how heavy mobile texters interpret the words “most people” in the general trust measurement. Heavy users assume that “most people” only refers to friends, family, and others going to the same school. When the effect of the “most people” assumption is controlled, the positive association between texting and general trust disappears. Further exploration of the data shows that heavy texting nevertheless has negative implications for social tolerance and social caution, both of which are theoretically proximate to general trust.     

Pattern synthesis of time‐modulated conformal arrays by an improved differential evolution algorithm

In this article, an improved differential evolution algorithm (IDE) based on two different colonies is proposed and applied to time‐modulated conformal arrays syntheses. The whole population of IDE is divided into two parts. The one part searches the solution globally while the other searches the neighborhood of the solution provided by the previous one. Benchmark functions are provided to testify IDE. Furthermore, IDE is applied to synthetize sum‐difference patterns with a 1 × 16 elements time‐modulated circular array and low sidelobe level (SLL) patterns with an 8 × 12 elements time‐modulated cone array. After optimization, the sideband level (SBL) of the circular array at the first sideband frequency is ?1.00 dB. The SLL and SBL at the first sideband frequency of the cone array are lower than ?30.00 and ?20.00 dB, respectively. Experiment results verify the superior performance of IDE. Moreover, to accelerate the computation speed, graphics processing unit parallel computing technique is introduced into pattern synthesis and the acceleration ratios of more than 23 times can be achieved. © 2014 Wiley Periodicals, Inc. Int J RF and Microwave CAE 24:697–705, 2014.     

Parallel Implementation of the BiCGStab（2） Method in GPU Using CUDA and Matlab for Solution of Linear Systems

This paper presents a parallel implementation of the hybrid BiCGStab（2） （bi-conjugate gradient stabilized） iterative method in a GPU （graphics processing unit） for solution of large and sparse linear systems. This implementation uses the CUDA-Matlab integration, in which the method operations are performed in a GPU core using Matlab built-in functions. The goal is to show that the exploitation of parallelism by using this new technology can provide a significant computational performance. For the validation of the work, we compared the proposed implementation with a BiCGStab（2） sequential and parallelized implementation in the C and CUDA-C languages. The results showed that the proposed implementation is more efficient and can be viable for simulations being carried out with quality and in a timely manner. The gains in computational efficiency were 76x and 6x compared to the implementation in C and CUDA-C, respectively.     

基于GPU-CUDA的共轭斜量法实现及性能对比

偏微分方程数值解法(包括有限差分法、有限元法)以及大量的数学物理方程数值解法最终都会演变成求解大型线性方程组。因此,探讨快速、稳定、精确的大型线性方程组解法一直是数值计算领域不断深入研究的课题且具有特别重要的意义。在迭代法中,共轭斜量法(又称共轭梯度法)被公认为最好的方法之一。但是,该方法最大缺点是仅适用于线性方程组系数矩阵为对称正定矩阵的情况,而且常规的CPU算法实现非常耗时。为此,通过将线性方程组系数矩阵作转换成对称矩阵后实施基于GPU-CUDA的快速共轭斜量法来解决一般性大型线性方程组的求解问题。试验结果表明:在求解效率方面,基于GPU-CUDA的共轭斜量法运行效率高,当线性方程组阶数超过3000时,其加速比将超过14;在解的精确性与求解过程的稳定性方面,与高斯列主元消去法相当。基于GPU-CUDA的快速共轭斜量法是求解一般性大型线性方程组快速而非常有效的方法。     

基于宇宙计算的图形处理器算法实现

下一代观测望远镜将会产生数以亿计的星系测量数据值,这将导致使用中央处理器处理数据时效率低下、成本较高。为了解决这一问题,提出了基于宇宙计算的图形处理器算法。研究了两点式角相关函数以及孔径质量统计这两种宇宙学的计算方法,构建算法代码,并使用统一计算设备架构在图形处理器上实现了这两种算法;比较了算法在中央处理器和图形处理器上使用的运行速度。实验结果表明,与中央处理器相比,使用图形处理器的计算速度得到了显著提高。     

混炼硅橡胶的配合技术(四)

介绍了几种胶辊用混炼硅橡胶的配制,包括低硬度胶辊用混炼硅橡胶、低环硅氧烷含量的胶辊用混炼硅橡胶、送纸胶辊用混炼硅橡胶及工业胶辊用混炼硅橡胶的配制。     

一种并行中英文混合多模式匹配算法

针对中英文混合文本的匹配准确性及大规模数据文本的匹配效率等问题,基于经典的线索化完全哈希特里树算法,提出一种并行化的中英文混合多模式文本匹配算法。采用拆分文本降低多模式匹配算法的串行度,进而在拆分出的小文本上并行地执行文本匹配。通过并行化预处理过程,设计新的存储结构。实验结果表明,该算法在保证结果正确的前提下,执行效率高于经典的串行匹配算法,当数据规模达到226个字符时,可以获得8倍以上的加速比。