MMNNN: A tree-based Multicast Mechanism for NoC-based deep Neural Network accelerators

Network-on-Chip (NoC) devices have been widely used in multiprocessor systems. In recent years, NoC-based Deep Neural Network (DNN) accelerators have been proposed to connect neural computing devices using NoCs. Such designs dramatically reduce off-chip memory accesses of these platforms. However, the large number of one-to-many packet transfers significantly degrade performance with traditional unicast channels. We propose a multicast mechanism for a NoC-based DNN accelerator called Multicast Mechanism for NoC-based Neural Network accelerator (MMNNN). To do so, we propose a tree-based multicast routing algorithm with excellent scalability and the ability to minimize the number of packets in the network. We also propose a router architecture for single-flit packets. Our proposed router transfers flits to multiple destinations in a single process and has no head-of-line blocking issue, offering higher throughput and lower latency than traditional wormhole router architectures. Simulation results show that our proposed multicast mechanism offers excellent performance in classification latency, average packet latency, and energy consumption.     

A hardware/software platform for QoS bridging over multi-chip NoC-based systems

Recent embedded systems integrate a growing number of intellectual property cores into increasingly large designs. Implementation, prototyping, and verification of such large systems has become very challenging. One of the reasons is that chips/FPGAs resources are limited and therefore it is not always possible to implement the whole design in the traditional system-on-a-chip solutions. The state-of-the-art is to partition such systems into smaller sub-systems to implement each on a separate chip. Consequently, it requires interconnecting separate chips/FPGAs. Since Networks-on-Chip (NoCs) have become common interconnection solutions in embedded designs, we propose to bridge NoC-based SoCs enabling a generic multi-chip systems interconnection. In this context, the contribution of this paper is threefold, (i) we explore the NoC protocol stack to determine the best layer for implementing the off-chip bridge, (ii) we propose a generic hardware architecture for the bridge, and (iii) we develop a new software architecture enabling seamless configuration and communication of multi-chip NoC-based SoCs. Finally, we demonstrate performance, i.e., bandwidth and latency, of the bridge in a multi-FPGA platform, while the bridge guarantees QoS of traffic. The synthesis results indicate the implementation area cost of the bridge is only 1% of Xilinx Virtex6 FPGA.     

Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network

Biologically-inspired packet switched network on chip (NoC) based hardware spiking neural network (SNN) architectures have been proposed as an embedded computing platform for classification, estimation and control applications. Storage of large synaptic connectivity (SNN topology) information in SNNs require large distributed on-chip memory, which poses serious challenges for compact hardware implementation of such architectures. Based on the structured neural organisation observed in human brain, a modular neural networks (MNN) design strategy partitions complex application tasks into smaller subtasks executing on distinct neural network modules, and integrates intermediate outputs in higher level functions. This paper proposes a hardware modular neural tile (MNT) architecture that reduces the SNN topology memory requirement of NoC-based hardware SNNs by using a combination of fixed and configurable synaptic connections. The proposed MNT contains a 16:16 fully-connected feed-forward SNN structure and integrates in a mesh topology NoC communication infrastructure. The SNN topology memory requirement is 50 % of the monolithic NoC-based hardware SNN implementation. The paper also presents a lookup table based SNN topology memory allocation technique, which further increases the memory utilisation efficiency. Overall the area requirement of the architecture is reduced by an average of 66 % for practical SNN application topologies. The paper presents micro-architecture details of the proposed MNT and digital neuron circuit. The proposed architecture has been validated on a Xilinx Virtex-6 FPGA and synthesised using 65 nm low-power CMOS technology. The evolvable capability of the proposed MNT and its suitability for executing subtasks within a MNN execution architecture is demonstrated by successfully evolving benchmark SNN application tasks representing classification and non-linear control functions. The paper addresses hardware modular SNN design and implementation challenges and contributes to the development of a compact hardware modular SNN architecture suitable for embedded applications     

支持存储访问的NoC模拟器的设计与实现

随着片上网络的发展,片上多处理器系统通信性能提高的同时,存储器的访问性能将成为片上多处理器系统的性能瓶颈.目前片上网络的研究主要依赖于模拟器,而现有的片上网络模拟器都不能完成对存储器访问的准确模拟.本文设计并实现了一个能对存储器访问进行模拟的模拟器,为存储器性能的研究提供了一个实验平台;论文通过采用大量访问集对该模拟器进行测试,得出了若干条与存储器访问性能优化相关的片上网络设计建议.     

LPNet: A DNN based latency prediction technique for application mapping in Network-on-Chip design

Analytical models used for latency estimation of Network-on-Chip (NoC) are not producing reliable accuracy. This makes these analytical models difficult to use in optimization of design space exploration. In this paper, we propose a learning based model using deep neural network (DNN) for latency predictions. Input features for DNN model are collected from analytical model as well as from Booksim simulator. Then this DNN model has been adopted in mapping optimization loop for predicting the best mapping of given application and NoC parameters combination. Our simulations show that using the proposed DNN model, prediction error is less than 12% for both synthetic and application specific traffic. More than 108 times speedup could be achieved using DPSO with DNN model compared to DPSO using Booksim simulator.     

基于多FPGA的NoC多核处理器验证平台设计

为了能够灵活地验证和实现自主设计的基于NoC的多核处理器,缩短NoC多核处理器的设计周期,提出了设计集成4片Virtex-6-550T FPGA的NoC多核处理器原型芯片设计/验证平台.分析和评估了NoC多核处理器的规模以及对FPGA硬件资源的需求,在此基础上给出了集成4片FPGA的开发板详细设计方案,并对各主要模块如互联架构、电源、板级时钟分布、接口技术、存储资源等关键设计要点进行阐述.描述了开发板各个主要模块的测试过程和结果,表明了该设计的可行性.     

A generic FPGA prototype for on-chip systems with network-on-chip communication infrastructure

As System-on-Chips (SoCs) grow in complexity and size, proposals of networks-on-chip (NoCs) as the on-chip communication infrastructure are justified by reusability, scalability, and energy efficiency provided by the interconnection networks. Simulation and mathematical analysis offer flexibility for the evaluations under various network configurations. However, the accuracy of such analyzing methods largely depends on the approximations made. On the other hand, prototyping can be used to improve the evaluation accuracy by bringing the design closer to reality. In this paper, we propose a FPGA prototype that is general enough to model different video-processing SoCs where different cores communicate via NoC. To model NoC, we accurately implement a fully-synthesized on-chip router supporting multiple virtual channels. For the processing nodes, on the other side, we propose a general and simple traffic generator capable of modeling different synthetic functions (i.e. Poisson and self-similar). Indeed, the application traffic is modeled using 1-D hybrid cellular automata which can effectively generate high quality pseudorandom patterns. Finally, for the energy efficiency, the proposed prototype is capable to support multiple frequency regions. To realize the voltage–frequency island partitioned SoC, we use the utilities that Xilinx FPGA platform offers to design Globally Synchronous Locally Asynchronous (GALS) systems via Delay-Locked Loop elements.     

A multi-processor NoC-based architecture for real-time image/video enhancement

The paper presents a multi-processor architecture for real-time and low-power image and video enhancement applications. Differently from other state-of-the-art parallel architectures the proposed solution is composed of heterogeneous tiles. The tiles have computational and memory capabilities, support different algorithmic classes and are connected by a novel Network-on-Chip (NoC) infrastructure. The proposed packet-switched data transfer scheme avoids communication bottlenecks when more tiles are working concurrently. The functional performances of the NoC-based multi-processor architecture are assessed by presenting the achieved results when the platform is programmed to support different enhancement algorithms for still images or videos. The implementation complexity of the NoC-based multi-tile platform, integrated in 65 nm CMOS technology, is reported and discussed.     

Power optimization for application-specific networks-on-chips: A topology-based approach

This paper analyzes the main sources of power consumption in Networks-on-Chip (NoC)-based systems. Analytical power models of global interconnection links are studied at different levels of abstraction. Additionally, power measurement experiments are performed for different types of routers. Based on this study, we propose a new topology-based methodology to optimize the power consumption of complex NoC-based systems at early design phases. The efficiency of the proposed methodology is verified through a case study of an MPEG4 video application. Experimental results show a promising improvement in power consumption (8.55%), average number of hops (10.80%), and number of global links (56.25%) compared to the best known related work.     

Random access schemes for efficient FPGA SpMV acceleration

Utilizing hardware resources efficiently is vital to building the future generation of high-performance computing systems. The sparse matrix – dense vector multiplication (SpMV) kernel, which is notorious for its poor efficiency on conventional processors, is a key component in many scientific computing applications and increasing SpMV efficiency can contribute significantly to improving overall system efficiency. The major challenge in implementing SpMV efficiently is handling the input-dependent memory access patterns, and reconfigurable logic is a strong candidate for tackling this problem via memory system customization. In this work, we consider three schemes (all off-chip, all on-chip, caching) for servicing the irregular-access component of SpMV and investigate their effects on accelerator efficiency. To combine the strengths of on-chip and off-chip random accesses, we propose a hardware-software caching scheme named NCVCS that combines software preprocessing with a nonblocking cache to enable highly efficient SpMV accelerators with modest on-chip memory requirements. Our results from the comparison of the three schemes implemented as part of an FPGA SpMV accelerator show that our scheme effectively combines the high efficiency from on-chip accesses with the capability of working with large matrices from off-chip accesses.     

一种新型片上网络互连结构的仿真和实现

综合性能、硬件实现等方面考虑,提出一种基于片上网络的互连拓扑结构-层次化路由结构MLR(Multi-Layer Router).该结构通过层次化设计减小网络直径,具有良好的对称性和扩展性.网络建模仿真和硬件实现结果显示,在不同网络负载和不同IP核节点数的情况下,MLR与传统结构相比,在处理网络通信时,对于网络丢包率、通信延迟和网络吞吐量等网络性能参数均有最多50%-70%的提升;同时通过共享路由的方式,减少了超过20%的芯片面积和40%以上的动态功耗,有效降低了互连结构的硬件开销     

CoQoS: Coordinating QoS-aware shared resources in NoC-based SoCs

Contention in performance-critical shared resources affects performance and quality-of-service (QoS) significantly. While this issue has been studied recently in CMP architectures, the same problem exists in SoC architectures where the challenge is even more severe due to the contention of shared resources between programmable cores and fixed-function IP blocks. In the SoC environment, efficient resource sharing and a guarantee of a certain level of QoS are highly desirable. Researchers have proposed different techniques to support QoS, but most existing works focus on only one individual resource. Coordinated management of multiple QoS-aware shared resources remains an open problem. In this paper, we propose a class-of-service based QoS architecture (CoQoS), which can jointly manage three performance-critical resources (cache, NoC, and memory) in a NoC-based SoC platform. We evaluate the interaction between the QoS-aware allocation of shared resources in a trace-driven platform simulator consisting of detailed NoC and cache/memory models. Our simulations show that the class-of-service based approach provides a low-cost flexible solution for SoCs. We show that assigning the same class-of-service to multiple resources is not as effective as tuning the class-of-service of each resource while observing the joint interactions. This demonstrates the importance of overall QoS support and the coordination of QoS-aware shared resources.     

Elastic superposition task mapping for NoC-based reconfigurable systems

With technology progress, more and more applications are integrated into a single chip. This requires a large number of processing elements (PEs) in a system, such that computation can be effectively enhanced through parallel processing. To support more efficient parallel processing, the Network-on-Chip (NoC) is being increasingly adopted as an interconnection architecture. Nevertheless, for NoC-based reconfigurable systems, the issue of mapping tasks to the PEs becomes more complex, due to the characteristic of hardware reconfiguration. This work proposes a novel Elastic Superposition Mapping (ESM) that introduces a useful PE reservation heuristic along with dynamic cross-application superposition. The ESM can provide a great elasticity for an NoC-based reconfigurable system to map more applications. Thus, the task load on PE will increase. Experiments show that, compared to the state-of-the-art mapping methods, 7% to 49% more applications can be executed, the average task load on PE can be increased by 5.5% to 56%, and the application waiting time can be reduced by 11% to 54%.     

面向应用的流存储系统评测与改进

有限的片外存储带宽是制约流处理器性能提升的瓶颈之一,流存储系统已经采用了多种方式来缓解这个问题,但当前的设计并没有充分考虑应用具体的访存模式对有效带宽利用率的影响.通过分析和实验,评估流存储系统主要设计参数对不同访存模式的优化效果;在此基础上针对不同的流访问并行度提出了相应的结构改进,加入宽发射和短作业优先调度支持,充分挖掘存储访问的局部性和并行性,改善了负载平衡,从而有效地提高了片外带宽的使用效率和流程序的整体性能.     

ProNoC: A low latency network-on-chip based many-core system-on-chip prototyping platform

Network-on-chip (NoC) is an emerging interconnect infrastructure to address the scalability limitation of conventional shared bus architecture for many-core system-on-chip (MCSoC). Current field-programmable gate arrays (FPGAs) have over million lookup tables, making it possible to prototype a complete NoC-based MCSoC on a single FPGA device. FPGA prototyping allows rapid system verification and optimum design parameters estimation. However, existing NoC-based MCSoC prototypes are usually adopting simple NoC architectural functionality. These NoC prototypes cannot represent a realistic projection of the state-of-the-art application-specific integrated circuit (ASIC) NoCs as these prototypes have limited overall system performance. This paper presents ProNoC, an integrated tool for rapid prototyping and validation of NoC-based MCSoC projects targeting FPGA devices. ProNoC adopts most advanced NoC features such as the support of virtual channel (VC), virtual network, low latency routing and different routing algorithms. Results show that NoC interconnect in ProNoC outperforms CONNECT, the most recent VC based prototype NoC with lower logic cell utilization, higher maximum operating frequency, higher average saturation throughput, and lower average communication latency. Moreover, ProNoC is equipped with graphical user interface to facilitate the development of MCSoC prototypes on FPGA platforms.     

New access modes of parallel memory subsystem for sub-pixel motion estimation

Accessing pixels in memory is a well-known bottleneck of SIMD (single instruction multiple data) processors in video/imaging. To tackle it, we propose new block and row access modes of parallel on-chip memory subsystem, which enable a higher processing throughput and lower energy consumption than the access modes of the state-of-the-art subsystems. The new access modes significantly reduce the number of on-chip memory accesses, and thereby accelerate one of key video/imaging kernels: sub-pixel block-matching motion estimation. The main idea is to exploit spatial overlaps of blocks/rows accessed for pixel interpolation, which are known at the subsystem design-time, and merge multiple accesses into a single one by accessing somewhat more pixels at a time than with other parallel memories. To avoid the need for a wider, and, therefore, more costly SIMD datapath, we propose new memory read operations that split all pixels accessed at a time into multiple SIMD-wide blocks/rows, in a convenient way for further processing. As a proof of concept, we describe a parametric, scalable, and cost-efficient architecture that supports the new access modes. The architecture is based on a previously proposed set of memory banks with multiple pixels per bank word, and a previously proposed shifted scheme for arranging pixels in the banks. We analytically and experimentally demonstrate advantages of this work on a case study of sub-pixel motion estimation for video frame-rate conversion. The implemented motion estimator processes 2160p video at 60 fps in real time, while clocked at 600 MHz. Compared to the implementations based on the state-of-the-art subsystems, this work enables 40–70 % higher throughput, consumes 17–44 % less energy and has similar silicon area and off-chip memory bandwidth costs. That is 1.8–2.9 times more efficient than the prior art, considering the throughput and all costs, i.e., consumption, area, and off-chip bandwidth. Such a higher efficiency is the result of the new access modes, which reduced the number of on-chip memory accesses by 1.6–2.1 times, and the cost-efficient architecture.     

Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks

Quantized neural networks (QNNs), which use low bitwidth numbers for representing parameters and performing computations, have been proposed to reduce the computation complexity, storage size and memory usage. In QNNs, parameters and activations are uniformly quantized, such that the multiplications and additions can be accelerated by bitwise operations. However, distributions of parameters in neural networks are often imbalanced, such that the uniform quantization determined from extremal values may underutilize available bitwidth. In this paper, we propose a novel quantization method that can ensure the balance of distributions of quantized values. Our method first recursively partitions the parameters by percentiles into balanced bins, and then applies uniform quantization. We also introduce computationally cheaper approximations of percentiles to reduce the computation overhead introduced. Overall, our method improves the prediction accuracies of QNNs without introducing extra computation during inference, has negligible impact on training speed, and is applicable to both convolutional neural networks and recurrent neural networks. Experiments on standard datasets including ImageNet and Penn Treebank confirm the effectiveness of our method. On ImageNet, the top-5 error rate of our 4-bit quantized GoogLeNet model is 12.7%, which is superior to the state-of-the-arts of QNNs.     

基于FPGA加速的低功耗的MobileNetV2网络识别系统

近年来,卷积神经网络由于其出色的性能被广泛应用在各个领域,如图像识别、语音识别与翻译和自动驾驶等;但是传统卷积神经网络（Convolutional Neural Network,CNN）存在参数多,计算量大,部署在CPU与GPU上推理速度慢、功耗大的问题。针对上述问题,采用量化感知训练（Quantization Aware Training,QAT）的方式在保证图像分类准确率的前提下,将网络参数总量压缩为原网络的1/4;将网络权重全部部署在FPGA的片内资源上,克服了片外存储带宽的限制,减少了访问片外存储资源带来的功耗;在MobileNetV2网络的层内以及相邻的点卷积层之间提出一种协同配合的流水线结构,极大的提高了网络的实时性;提出一种存储器与数据读取的优化策略,根据并行度调整数据的存储排列方式及读取顺序,进一步节约了片内BRAM资源。最终在Xilinx的Virtex-7 VC707开发板上实现了一套性能优、功耗小的轻量级卷积神经网络MobileNetV2识别系统,200HZ时钟下达到了170.06 GOP/s的吞吐量,功耗仅为6.13W,能耗比达到了27.74 GOP/s/W,是CPU的92倍,GPU的25倍,性能较其他实现有明显的优势。     

Adaptive image compression technique for wireless sensor networks

When using wireless sensor networks for real-time image transmission, some critical points should be considered. These points are limited computational power, storage capability, narrow bandwidth and required energy. Therefore, efficient compression and transmission of images in wireless sensor network is considered. To address the above mentioned concerns, an efficient adaptive compression scheme that ensures a significant computational and energy reduction as well as communication with minimal degradation of the image quality is proposed. This scheme is based on wavelet image transform and distributed image compression by sharing the processing of tasks to extend the overall lifetime of the network. Simulation results are presented and they show that the proposed scheme optimizes the network lifetime, reduces significantly the amount of the required memory and minimizes the computation energy by reducing the number of arithmetic operations and memory accesses.