基于模板匹配的机器视觉话机检测系统

针对流水线产品的自动检测需求的不断提升,使用模板匹配算法实现智能话机检测设备．并进行软件仿真,该系统用于检测话机按键的错装、漏装以及字符印刷错误等问题,实时获取产品图像和检测产品,可视化效果好。     

An efficient online citrus counting system for large-scale unstructured orchards based on the unmanned aerial vehicle

The accurate detection and counting of fruits in natural environments are key steps for the early yield estimation of orchards and the realization of smart orchard production management. However, existing citrus counting algorithms have two primary limitations: the performance of counting algorithms needs to be improved, and their system operation efficiency is low in practical applications. Therefore, in this paper, we propose a novel end-to-end orchard fruit counting pipeline that can be used by multiple unmanned aerial vehicles (UAVs) in parallel to help overcome the above problems. First, to obtain on-board camera images online, an innovative UAV live broadcast platform was developed for the orchard scene. Second, for this challenging specific scene, a detection network named Citrus-YOLO was designed to detect fruits in the video stream in real-time. Then, the DeepSort algorithm was used to assign a specific ID to each citrus fruit in the online UAV scene and track the fruits across video frames. Finally, a nonuniform distributed counter was proposed to correct the fruit count during the tracking process, and this can significantly reduce the counting errors caused by tracking failure. This is the first work to realize online and end-to-end counting in a field orchard environment. The experimental results show that the F1 score and mean absolute percentage error of the method are 89.07% and 12.75%, respectively, indicating that the system can quickly and accurately achieve fruit counting in large-scale unstructured citrus orchards. Although our work is discussed in the context of fruit counting, it can be extended to the detection, tracking and counting of a variety of other objects of interest in UAV application scenarios     

基于数据驱动的管道云边协同泄漏检测方法

随着管道运输规模的不断扩大和泄漏检测流程的日趋复杂,管道泄漏检测系统中数据采集、传输和处理等任务难度呈几何级数上升.鉴于此,针对基于数据驱动的管道云边协同泄漏检测方法展开研究,首先针对系统在数据获取方面中压力数据采集量大、数据之间存在冗余的问题,提出一种自适应数据压缩与采集算法;然后依据云边协同调度策略的需求,对云边协同系统中各个环节进行任务细粒度划分,并根据划分后子任务的计算时延和传输时延提出云边协同下管道泄漏检测系统的任务拓扑模型;最后将系统的优化目标定义为在任务执行时间限制下的边缘控制器利用率,进而通过遗传算法求解时间限制下的最优调度策略.仿真分析验证了管道云边协同泄漏检测方法的有效性,所提出方法可以实现管道泄漏事件快速报警.     

基于改进Faster RCNN的工业机器人分拣系统

传统的分拣作业无法伴随工作环境的变化进行相应的调整,针对此种不足,出现了基于机器视觉的分拣机器人的相关研究,通过将图像处理和特征工程技术引入视觉模块,使得分拣系统能适时的调整.不同于这些方法,本研究基于实验室的工业分拣系统,将深度学习方法应用其中.通过将Faster RCNN检测算法引入视觉模块并对区域提取网络RPN进行相关改进,加快Faster RCNN模型的检测过程,使得该系统满足工业的实时性要求.Faster RCNN作为一种端到端的方法,能自动对输入图像生成更具表达力的特征,对相应目标提取相应特征,这避免了人工设计特征,它的特征自动生成能力使其能适用于各种场景,这提升了工业分拣机器人的环境适应能力.     

Delay composition in preemptive and non-preemptive real-time pipelines

Uniprocessor schedulability theory made great strides, in part, due to the simplicity of composing the delay of a job from the execution times of higher-priority jobs that preempt it. In this paper, we bound the end-to-end delay of a job in a multistage pipeline as a function of job execution times on different stages under preemptive as well as non-preemptive scheduling. We show that the end-to-end delay is bounded by that of a single virtual “bottleneck” stage plus a small additive component. This contribution effectively transforms the pipeline into a single stage system. The wealth of schedulability analysis techniques derived for uniprocessors can then be applied to decide the schedulability of the pipeline. The transformation does not require imposing artificial per-stage deadlines, but rather models the pipeline as a whole and uses the end-to-end deadlines directly in the single-stage analysis. It also does not make assumptions on job arrival patterns or periodicity and thus can be applied to periodic and aperiodic tasks alike. We show through simulations that this approach outperforms previous pipeline schedulability tests except for very short pipelines or when deadlines are sufficiently large. The reason lies in the way we account for execution overlap among stages. We discuss how previous approaches account for overlap and point out interesting differences that lead to different performance advantages in different cases. Further, we also show that in certain cases non-preemptive scheduling can result in higher system utilization than preemptive scheduling in pipelined systems. We hope that the pipeline delay composition rule, derived in this paper, may be a step towards a general schedulability analysis foundation for large distributed systems.

End-to-end deadline control for aperiodic tasks in distributed real-time systems

A category of Distributed Real-Time Systems (DRTS) that has multiprocessor pipeline architecture is increasingly used. The key challenge of such systems is to guarantee the end-to-end deadlines of aperiodic tasks. This paper proposes an end-to-end deadline control model, called Linear Quadratic Stochastic Optimal Control Model (LQ-SOCM), which features a distributed feedback control that dynamically enforces the desired performance. The control system considers the aperiodic task arrivals and execution times’ variation as the two external factors of the system unpredictability. LQ-SOCM uses discrete time state space equation to describe the real-time computing system. Then, in the actuator design, a continuous manner is adopted to deal with discrete QoS (Quality of Service) adaptation. Finally, experiments demonstrate that the system is globally stable and can statistically provide the end-to-end deadline guarantee for aperiodic tasks. At the same time, LQ-SOCM is capable of effectively improving the system throughput.

Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing

Quality and efficiency are crucial indicators of any manufacturing company. Many companies are suffering from a shortage of experienced workers across the production line to perform complex assembly tasks. To reduce time and error in an assembly task, a worker-centered system consisting of multi-modal Augmented Reality (AR) instructions with the support of a deep learning network for tool detection is introduced. The integrated AR is designed to provide on-site instructions including various visual renderings with a fine-tuned Region-based Convolutional Neural Network, which is trained on a synthetic tool dataset. The dataset is generated using CAD models of tools and displayed onto a 2D scene without using real tool images. By experimenting the system to a mechanical assembly of a CNC carving machine, the result of a designed experiment shows that the system helps reduce the time and errors of the given assembly tasks by 33.2 % and 32.4 %, respectively. With the integrated system, an efficient, customizable smart AR instruction system capable of sensing, characterizing requirements, and enhancing worker’s performance has been built and demonstrated.     

3D visual perception for self-driving cars using a multi-camera system: Calibration,mapping, localization,and obstacle detection

Cameras are a crucial exteroceptive sensor for self-driving cars as they are low-cost and small, provide appearance information about the environment, and work in various weather conditions. They can be used for multiple purposes such as visual navigation and obstacle detection. We can use a surround multi-camera system to cover the full 360-degree field-of-view around the car. In this way, we avoid blind spots which can otherwise lead to accidents. To minimize the number of cameras needed for surround perception, we utilize fisheye cameras. Consequently, standard vision pipelines for 3D mapping, visual localization, obstacle detection, etc. need to be adapted to take full advantage of the availability of multiple cameras rather than treat each camera individually. In addition, processing of fisheye images has to be supported. In this paper, we describe the camera calibration and subsequent processing pipeline for multi-fisheye-camera systems developed as part of the V-Charge project. This project seeks to enable automated valet parking for self-driving cars. Our pipeline is able to precisely calibrate multi-camera systems, build sparse 3D maps for visual navigation, visually localize the car with respect to these maps, generate accurate dense maps, as well as detect obstacles based on real-time depth map extraction.     

Aerial image dehazing using a deep convolutional autoencoder

Aerial images and videos are extensively used for object detection and target tracking. However, due to the presence of thin clouds, haze or smoke from buildings, the processing of aerial data can be challenging. Existing single-image dehazing methods that work on ground-to-ground images, do not perform well on aerial images. Moreover, current dehazing methods are not capable for real-time processing. In this paper, a new end-to-end aerial image dehazing method using a deep convolutional autoencoder is proposed. Using the convolutional autoencoder, the dehazing problem is divided into two parts, namely, encoder, which aims extract important features to dehaze hazy regions and decoder, which aims to reconstruct the dehazed image using the down-sampled image received from the encoder. In this proposed method, we also exploit the superpixels in two different scales to generate synthetic thin cloud data to train our network. Since this network is trained in an end-to-end manner, in the test phase, for each input hazy aerial image, the proposed algorithm outputs a dehazed version without requiring any other information such as transmission map or atmospheric light value. With the proposed method, hazy regions are dehazed and objects within hazy regions become more visible while the contrast of non-hazy regions is increased. Experimental results on synthetic and real hazy aerial images demonstrate the superiority of the proposed method compared to existing dehazing methods in terms of quality and speed.

     

Framework and deployment of a cloud-based advanced planning and scheduling system

Many small and medium-sized manufacturing enterprises (SMEs) have already implemented enterprise resource planning (ERP) and manufacturing execution system (MES) and began to start the journey of cloud manufacturing; however, the high cost of hardware and software investment, implementation, and maintenance usually hinder SMEs from adopting an advanced planning and scheduling (APS) system. This paper aims to develop a cloud-based APS (C-APS) system framework, the service structure, and approach of deploying the C-APS system in a public cloud infrastructure platform and service provider or hybrid cloud platform. The package diagram is proposed for building the C-APS system's virtual factory model to improve modeling efficiency and data stability. The C-APS system is a cloud-based and object-oriented software; its simulation-based scheduling engine can generate the significant production and operations schedule, and has the characteristics of on-demand self-service, quickly expanding and adjusting to the virtual plant model. The C-APS system's application in a leading automotive part assembly company's printed circuit board production scheduling shows that the input planning data model is easy to maintain. The scheduling quality is high; the computing time is short and acceptable for practical application.     

基于面结构光的机械工件三维扫描系统设计

针对工业机器人对自动化装配过程生产效率的提高以及工件拾取对三维扫描技术的应用需求,设计了能够准确提取机械工件表面点云的视觉系统。扫描系统主要由计算机、投影仪和工业相机构成。基于光学测量和机器视觉的原理,主要研究设计了扫描系统工业相机和投影仪的标定策略、结构光栅编码解码的检测策略以及点云重构的几何策略。对于机械工件三维扫描重构的多余背景平面点云,研究设计了通过随机选取点云并反复迭代构造背景平面实现分割的有效方法。实验结果表明采用面结构光技术,由投影仪投影不同频率的结构光栅在机械工件上,工业相机同步采集被机械工件调制的结构光栅图像,对图像中的光栅条纹进行提取并计算,并利用三角检测算法提取机械工件表面点云的方案具有高准确性,能够有效重构机械工件表面点云。     

极轨气象卫星图像的云自动检测方法研究

图像上云的存在给遥感图像的处理与分析带来了不便。本文探讨了利用极轨气象卫星图像可见光波段的反射率和热红外波段的亮温信息进行云自动检测的方法。该方法在可见光波段是否有效判断的基础上,进行图像水陆区域划分,然后对不同图像、不同波段的水域和陆域分别设定合理的阈值,实现了云自动检测目的。试验结果表明,该方法能够取得较高的云检测精度。     

Improving robustness of industrial object detection by automatic generation of synthetic images from CAD models

Object detection (OD) is used for visual quality control in factories. Images that compose training datasets are often collected directly from the production line and labeled with bounding boxes manually. Such data represent well the inference context but might lack diversity, implying a risk of overfitting. To address this issue, we propose a dataset construction method based on an automated pipeline, which receives a CAD model of an object and returns a set of realistic synthetic labeled images (code publicly available). Our approach can be easily used by non-expert users and is relevant for industrial applications, where CAD models are widely available. We performed experiments to compare the use of datasets obtained by the two different ways—collecting and labeling real images or applying the proposed automated pipeline—in the classification of five different industrial parts. To ensure that both approaches can be used without deep learning expertise, all training parameters were kept fixed during these experiments. In our results, both methods were successful for some objects but failed for others. However, we have shown that the combined use of real and synthetic images led to better results. This finding has the potential to make industrial OD models more robust to poor data collection and labeling errors, without increasing the difficulty of the training process.     

DEUCON: Decentralized End-to-End Utilization Control for Distributed Real-Time Systems

Many real-time systems must control their CPU utilizations in order to meet end-to-end deadlines and prevent overload. Utilization control is particularly challenging in distributed real-time systems with highly unpredictable workloads and a large number of end-to-end tasks and processors. This paper presents the decentralized end-to-end utilization control (DEUCON) algorithm, which can dynamically enforce the desired utilizations on multiple processors in such systems. In contrast to centralized control schemes adopted in earlier works, DEUCON features a novel decentralized control structure that requires only localized coordination among neighbor processors. DEUCON is systematically designed based on advances in distributed model predictive control theory. Both control-theoretic analysis and simulations show that DEUCON can provide robust utilization guarantees and maintain global system stability despite severe variations in task execution times. Furthermore, DEUCON can effectively distribute the computation and communication cost to different processors and tolerate considerable communication delay between local controllers. Our results indicate that DEUCON can provide a scalable and robust utilization control for large-scale distributed real-time systems executing in unpredictable environments.     

基于三维空间模型的特高压耐张塔跳线间隙检测方法

提出一种基于三维空间模型的特高压耐张塔跳线间隙检测方法研究,利用三维激光扫描数据建立特高压耐张塔整体空间扫描模型,并描述激光扫描建模的原理。据此确定出了待检测标的物的三维点云数据集合;通过预处理的方式去除点云数据检测系统内部噪声和环境噪声,对点云数据集合进行高斯平滑滤波、坐标转换和点云数据拼接,并输出空间点云数据的文件集合;获取的跳线检测测量结果中还有可能包含粗差、系统误差和偶然性误差,利用检测数据分析前必须消除各种误差因素的影响。实验数据表明提出方法的坐标检测精度值能够控制在+-1,并且总体的检测残差分布也位于合理的区间范围之内,实际检测效果优于传统检测方法。     

基于虚拟现实技术的移动机器人视觉伺服控制系统设计

为了有效确保移动机器人视觉伺服控制效果,提高移动机器人视觉伺服控制精度,设计了基于虚拟现实技术的移动机器人视觉伺服控制系统。通过三维视觉传感器和立体显示器等虚拟环境的I/O设备、位姿传感器、视觉图像处理器以及伺服控制器元件,完成系统硬件设计。从运动学和动力学两个方面,搭建移动机器人数学模型,利用标定的视觉相机,生成移动机器人实时视觉图像,通过图像滤波、畸变校正等步骤,完成图像的预处理。利用视觉图像,构建移动机器人虚拟移动环境。在虚拟现实技术下,通过目标定位、路线生成、碰撞检测、路线调整等步骤,规划移动机器人行动路线,通过控制量的计算,实现视觉伺服控制功能。系统测试结果表明,所设计控制系统的位置控制误差较小,姿态角和移动速度控制误差仅为0.05°和0.12m/s,移动机器人碰撞次数较少,具有较好的移动机器人视觉伺服控制效果,能够有效提高移动机器人视觉伺服控制精度。     

Digital twins to enhance the integration of ergonomics in the workplace design

Recently, human-centered design has become one of the most promising approaches for improving the entire production process design. During the design phase, among the main important aspects to investigate, ergonomic performance of the workplace (WP) plays a key role. It is well known that design errors can lead to significant delays in the design and engineering of a production process, especially when it is related to a complex system such as the assembly line of an automotive industry. Prediction of the ergonomic performance, which is often coarsely considered during the design phase, can represent a fundamental step in preventing ergonomic issues since the early design phase of a production process, avoiding also negative consequences on line balancing. Based on a concurrent engineering (CE) approach, the aim of this paper is to present a framework that uses digital twins of stations in order to minimise the time necessary to develop and design a new assembly line. The application of this procedure will allow avoiding the possibility of realising a line that reveals ergonomic problems and correcting design errors during the design phase and not just during the production phase. In this way, it is possible to achieve great advantages in terms of cost avoidance for the correction of the design errors and in terms of time to market, which will be significantly reduced. A digital twin of a real station of a Fiat Chrysler Automobiles (FCA) assembly line is presented to validate the numerical procedure and the design approach proposed in this paper. Finally, numerical results, regarding the evaluation of an ergonomic index, were compared with experimental ones achieved by analysing data collected during an experimental session.     

Performance analysis of opto-mechatronic image stabilization for a compact space camera

The paper presents new performance results for the enhanced concept of an opto-mechatronic camera stabilization assembly consisting of a high-speed onboard optical processor for real-time image motion measurement and a 2-axis piezo-drive assembly for high precision positioning of the focal plane assembly. The proposed visual servoing concept allows minimizing the size of the optics and the sensitivity to attitude disturbances. The image motion measurement is based on 2D spatial correlation of sequential images recorded from an in situ motion matrix sensor in the focal plane of the camera. The demanding computational requirements for the real-time 2D-correlation are covered by an embedded optical correlation processor (joint transform type). The paper presents briefly the system concept and fundamental working principles and it focuses on a detailed performance and error analysis of the image motion tracking subsystem. Simulation results of the end-to-end image motion compensation performance and first functional hardware-in-the-loop test results conclude the paper.     

基于磁致伸缩器扫频振动的禽蛋检测流水线系统

针对禽蛋运输及储存过程中造成裂纹、散黄等品质问题,设计一种基于磁致伸缩器扫频振动的禽蛋品质检测流水线系统,主要包括机械结构、传感与控制电路以及上位机软件系统.其中驱动电路控制流水线运行,到位检测传感器检测到禽蛋后控制磁致伸缩器对禽蛋进行扫频激振,同时通过麦克风采集禽蛋振动声音,并对采集的音频信号进行Yule-Walker功率谱分析和广义回归神经网络(GRNN,generalized regression neural network)建模,基于LabWindows/CVI开发的上位机软件系统实现禽蛋检测流水线的测控及禽蛋品质辨识和分类.实验表明,所设计的基于磁致伸缩器扫频振动的禽蛋检测流水线系统能够在线高效检测正常蛋、裂纹蛋和散黄蛋,检测速度约1枚/s.     

Data-driven dynamic bottleneck detection in complex manufacturing systems

Production (throughput) bottlenecks are the critical stations defining and constraining the overall productivity of a system. Effective and timely identification of bottlenecks provide manufacturers essential decision input to allocate limited maintenance and financial resources for throughput improvement. However, identifying throughput bottleneck in industry is not a trivial task. Bottlenecks are usually non-static (shifting) among stations during production, which requires dynamic bottleneck detection methods. An effective methodology requires proper handling of real-time production data and integration of factory physics knowledge. Traditional data-driven bottleneck detection methods only focus on serial production lines, while most production lines are complex. With careful revision and examination, those methods can hardly meet practical industrial needs. Therefore, this paper proposes a systematic approach for bottleneck detection for complex manufacturing systems with non-serial configurations. It extends a well-recognized bottleneck detection algorithm, “the Turning Point Method”, to complex manufacturing systems by evaluating and proposing appropriate heuristic rules. Several common industrial scenarios are evaluated and addressed in this paper, including closed loop structures, parallel line structures, and rework loop structures. The proposed methodology is demonstrated with a one-year pilot study at an automotive powertrain assembly line with complex manufacturing layouts. The result has shown a successful implementation that greatly improved the bottleneck detection capabilities for this manufacturing system and led to an over 30% gain in Overall Equipment Effectiveness (OEE).