A family of experiments to investigate the effects of groupware for software inspection

It is widely accepted that the inspection of software artifacts can find defects early in the development process and gather information on the quality of the evolving product. However, the inspection process is resource-intensive and involves tedious tasks, such as searching, sorting, and checking. Tool support for inspections can help accelerating these tasks and allows inspectors to concentrate on tasks particularly needing human attention. Only few tools are available for inspections. We have thus developed a set of groupware tools for both individual defect detection and inspection meetings to lower the effort of inspections and to increase their efficiency. This paper presents the Groupware-supported Inspection Process (GrIP) and describes tools for inspecting software requirements. As only little empirical work exists that directly compares paper-based and tool-based software inspection, we conducted a family of experiments in an academic environment to empirically investigate the effect of tool support regarding defect detection and inspection meetings. The main results of our family of experiments regarding individual defect detection are promising: The effectiveness of inspectors and teams is comparable to paper-based inspection without tool support; the inspection effort and defect overlap decreases significantly with tool support, while the efficiency of inspection teams increases considerably. Regarding tool support for inspection meetings the main findings of the experiments are that tool support considerably lowers the meeting effort, supports inspectors in identifying false positives, and reduces the number of true defects lost during a meeting. The number of unidentified false positives is still quite high.     

A software inspection process definition language and prototype support tool

Software inspection is a widely used method for finding defects in all types of software development documents. Many process variations exist, each designed for use under certain circumstances or to address some perceived deficiency in other methods. A desirable attribute of inspection is rigour, allowing the use of historical data to predict future performance and to suggest process improvements. Recent work in tool support for inspection is designed to tackle the issue of enforcing rigorous inspection, but these tools concentrate on enforcing a single, usually proprietary, method. This paper investigates existing inspection methods and derives a generic inspection process which can be used to describe any of these methods. This process is then used to determine a notation for describing any inspection process, which can consequently be used as input to an inspection support tool, allowing the support of any inspection method. The paper also demonstrates a system which uses the language to provide support for multiple inspection processes, and describes other desirable features of such a tool. © 1997 by John Wiley & Sons, Ltd.     

A migration strategy for automating a vision-based concentricity inspection station

This paper presents the development of an automation migration strategy in transforming a manual visual inspection work cell into a semi-automated one for a medical device manufacturer in China. The object under study is a washer/magnet subassembly used in an air release valve. These two circular components must be bonded concentrically and then inspected with bare eyes by a human inspector. Such inspection process was prone to error as the inspector struggled to keep up with the production cycle time. The methodology employed in this research consists of four steps. First, we examined the cost of rework through the Pareto analysis. The results indicated that the washer/magnet misalignment accounted for more than 40% of valve defects and thus deserved immediate attention. Next, we conducted two Kappa analyses to evaluate repeatability and reproducibility of the human inspectors assigned to perform the inspection tasks. The results showed that the human inspectors failed to pass these tests and a suitable automation solution must be sought. Afterwards, efforts were made to develop a vision based semi-automated concentricity inspection station to eliminate human inspection errors. Hardware setup, software algorithms, lighting and other supporting devices are presented in this paper as well as potential savings for such an installment. Finally, we conducted an economical analysis to compare the semi-automated solution with a fully automated one to identify the best automation migration strategy. The analysis results showed that the semi-automated solution was a favorable choice due to a shorter payback period and its ease of reinstallation if the factory is to be relocated.     

The Impact of Educational Background on the Effectiveness of Requirements Inspections: An Empirical Study

While the inspection of various software artifacts increases the quality of the end product, the effectiveness of an inspection depends largely on the individual inspectors involved. To address that issue, a large-scale controlled inspection experiment with over 70 professionals was conducted at Microsoft Corporation that focused on the relationship between an inspector's background and their effectiveness during a requirements inspection. The results of the study showed that inspectors with university degrees in majors not related to computer science found significantly more defects than those with degrees in computer science majors. We also observed that level of education (Masters, PhD), prior industrial experience or other job related experiences did not significantly impact the effectiveness of an inspector. The only other type of experience that had a significant impact on effectiveness was experience in writing requirements, i.e. professionals with prior experience writing requirements found statistically significant more defects than their counterparts.     

Analysis of visual and tactile search in industrial inspection

Abstract

A video system was developed to record the actions of inspectors during a visual and tactile inspection task in an automobile line plant. The system consisted of two video cameras and recorders operated synchronously. The recordings were analysed with a video motion analyser. The search behaviour of the inspectors was described by using four items of information derived from each video frame. As a consequence, the inspection task was resolved into elements with a discrimination of 1/30 s. Comparisons were made between the behaviour when flaws were deleted, missed and when there were no flaws and the search strategy required to locate flaws in the product was described. The recommended values of search parameters were estimated and this recommended search strategy was presented in a form understandable to the inspectors.     

A dynamic capture–recapture model for software defect prediction

Several models have been developed that attempt to predict the total number of defects in a software product. One such approach uses the capture–recapture model, a technique employed by biologists for predicting wildlife populations. In this method once the software is built and defects begin to be identified a prediction can be made for the total number of software defects present. But capture–recapture models rely on expert inspectors and the technique cannot be employed once the software has been released. The work reported here extends the capture–recapture technique to the post-inspection phase and to where inspection data is unavailable, by using user defect reports. The proposed technique does not rely on expert inspectors and is particularly suitable for open source software.     

Improving SEM Inspection Performance in Semiconductor Manufacturing Industry

Scanning electron microscope (SEM) station is one of the major quality control tasks in wafer manufacturing process. During the process of examining defects on a screen for a certain period of time, SEM inspectors frequently complain about visual fatigue problems. The purpose of the study was to evaluate the effect of wafer‐coating condition (noncoating treatment and gold‐coating treatment) and liquid crystal display (LCD) monitor size (14‐inch and 19‐inch monitor) from the aspects of objective visual fatigue measures (NPA and CFF), subjective eye fatigue rating, and inspection performance. Twelve SEM inspectors participated in this experiment. The results indicated that the treatment of gold‐coating wafer showed significant difference on reducing objective visual fatigue in NPA and CFF change. Using a 19‐inch LCD monitor also demonstrated the effect on reducing eye fatigue as well as improving inspection performance. The improvement of using a gold‐coating treatment wafer and a 19‐inch LCD monitor to reduce SEM inspectors' visual fatigue and to increase inspection performance is thus recommended. © 2012 Wiley Periodicals, Inc.     

Anomaly detection of defects on concrete structures with the convolutional autoencoder

This paper reports the application of deep learning for implementing the anomaly detection of defects on concrete structures, so as to facilitate the visual inspection of civil infrastructure. A convolutional autoencoder was trained as a reconstruction-based model, with the defect-free images, to rapidly and reliably detect defects from the large volume of image datasets. This training process was in the unsupervised mode, with no label needed, thereby requiring no prior knowledge and saving an enormous amount of time for label preparation. The built anomaly detector favors minimizing the reconstruction errors of defect-free images, which renders high reconstruction errors of defects, in turn, detecting the location of defects. The assessment shows that the proposed anomaly detection technique is robust and adaptable to defects on wide ranges of scales. Comparison was also made with the segmentation results produced by other automatic classical methods, revealing that the results made by the anomaly map outperform other segmentation methods, in terms of precision, recall, F₁ measure and F₂ measure, without severe under- and over-segmentation. Further, instead of merely being a binary map, each pixel of the anomaly map is represented by the anomaly score, which acts as a risk indicator for alerting inspectors, wherever defects on concrete structures are detected.     

Automated thermal fuse inspection using machine vision and artificial neural networks

Machine vision is an excellent tool for inspecting a variety of items such as textiles, fruit, printed circuit boards, electrical components, labels, integrated circuits, machine tools, etc. This paper presents an intelligent system that incorporates machine vision with artificial intelligent networks to automatically inspect thermal fuses. An effective inspection flow is proposed to detect four commonly seen defects, including black-dot, small-head, bur, and flake during the production of thermal fuses. Backpropagation neural networks and learning vector quantization performance is compared in detecting the bur defect because of its illegibility. Different numbers of defective samples were screened out from a production line in a case study company and used to demonstrate the efficacy of the proposed system. Currently, the proposed inspection system is operating at the case study company, replacing four to six human inspectors. The system not only ensures the quality of the thermal fuses produced, but also reduced the cost of manual visual inspection.     

From ethnography on infrastructure management to initial user feedback on PlaceMemo

This paper reports design requirements derived from an ethnographic fieldwork on road inspectors, the design of a mobile service supporting infrastructure management, and initial user feedback. Seeing that the road inspectors are truly mobile, acting in vast settings full of objects with variable status, they lack overview of their upcoming tasks. The PlaceMemo prototype allows them to place geographically coupled voice annotations in their work setting. The voice memos are played through their entire length before reaching the precise position of the recording, and are also accessible when being geographically distant. By handing out the prototype to a number of users, we successfully demonstrate that this lightweight context-aware system facilitates their mobile work. They obtain an overview supporting the advance preparation of inspection tours, and can easily access earlier recordings while driving, in order to be reminded and to prepare stops in time.     

A Comparison of Tool-Based and Paper-Based Software Inspection

Software inspection is an effective method of defect detection. Recent research activity has considered the development of tool support to further increase the efficiency and effectiveness of inspection, resulting in a number of prototype tools being developed. However, no comprehensive evaluations of these tools have been carried out to determine their effectiveness in comparison with traditional paper-based inspection. This issue must be addressed if tool-supported inspection is to become an accepted alternative to, or even replace, paper-based inspection. This paper describes a controlled experiment comparing the effectiveness of tool-supported software inspection with paper-based inspection, using a new prototype software inspection tool known as ASSIST (Asynchronous/Synchronous Software Inspection Support Tool). 43 students used ASSIST and paper-based inspection to inspect two C++ programs of approximately 150 lines. The subjects performed both individual inspection and a group collection meeting, representing a typical inspection process. It was found that subjects performed equally well with tool-based inspection as with paper-based, measured in terms of the number of defects found, the number of false positives reported, and meeting gains and losses.     

基于神经网络的冷轧带钢表面缺陷检测

带钢表面缺陷是影响带钢质量的重要因素,对带钢进行表面缺陷检测对提高带钢质量具有重要意义。传统人工检测的方法往往不能得到令人满意的检测结果。为此,提出了采用基于前馈神经网络(FFN)的方法对在线带钢的表面缺陷进行检测,检测结果令人满意,表明了该方法的有效性。     

A comprehensive evaluation of capture-recapture models forestimating software defect content

An important requirement to control the inspection of software artifacts is to be able to decide, based on more objective information, whether the inspection can stop or whether it should continue to achieve a suitable level of artifact quality. A prediction of the number of remaining defects in an inspected artifact can be used for decision making. Several studies in software engineering have considered capture-recapture models to make a prediction. However, few studies compare the actual number of remaining defects to the one predicted by a capture-recapture model on real software engineering artifacts. The authors focus on traditional inspections and estimate, based on actual inspections data, the degree of accuracy of relevant state-of-the-art capture-recapture models for which statistical estimators exist. In order to assess their robustness, we look at the impact of the number of inspectors and the number of actual defects on the estimators' accuracy based on actual inspection data. Our results show that models are strongly affected by the number of inspectors, and therefore one must consider this factor before using capture-recapture models. When the number of inspectors is too small, no model is sufficiently accurate and underestimation may be substantial. In addition, some models perform better than others in a large number of conditions and plausible reasons are discussed. Based on our analyses, we recommend using a model taking into account that defects have different probabilities of being detected and the corresponding Jackknife Estimator. Furthermore, we calibrate the prediction models based on their relative error, as previously computed on other inspections. We identified theoretical limitations to this approach which were then confirmed by the data     

ON THE ACCURACY OF INSPECTORS

Reasons for inspector inaccuracy have been examined in three main groups: reasons of basic individual abilities; of formal organization (training, instructions, physical conditions, the lay-out of the job); and of interpersonal relations and social relations. Without denying the importance of the basic individual abilities of inspectors, which must set the ultimate limits of accuracy, it seems that the actual limits in a working situation are set by the other two groups of reasons. These practical limits may be well inside the limits set by basic psychological and physiological functions. For example, given an inspector who is well equipped with the basic abilities and aptitudes for the actual inspection task itself, he cannot operate more accurately than his instructions, for instance, allow him to. Moreover, even if well selected, well trained and well briefed, he still can be no more accurate than the pressures of interpersonal and other social relations permit. These interpersonal relations do not necessarily make him pass work that should be failed; they may also make him fail work that should be passed. Conversely, the production man will be more eager to ‘ get products past ’, to trick the inspector, when he does not‘ sanction ’ the inspector. This may in turn make the inspector reject more of the work of the man who tries to trick him than is actually bad. These interpersonal and social relations become all the more important when the inspection task is the more ‘ socio-technical ’—involving direct interaction with production.

It seems that when relations between production and inspection are poor. when production feel they can not sanction the inspectors and/or their standards, and when inspection chooses to play its role in a dominant, authoritative, and essentially invidious way. rather than as finding neutral facts, then not only will inspector accuracy be adversely affected, but there will also be strong pressures against inspection supervisors testing their inspector's accuracy.

Although the obvious man to check inspector accuracy is the inspector's own supervisor, there are powerful pressures against his doing this,Not only are there practical difficulties, but also there is the fact that inspection supervisors, and others, tend to become ‘ product-bound ’, even to the neglect of their true supervisory duties. Moreover, the pressures of interdepartmental relations encourage inspection supervisors to see themselves first as inspectors, and only secondarily as supervisors.

The general conclusion is that inspector accuracy, in a working situation, is determined by a wide range of factors. Problems of inaccuracy must then be studied in a wider context than is given by any single approach.     

An advanced auto-inspection system for micro-router collapse

In this paper, we propose an auto-optical inspection (AOI) system that can inspect micro-router (router) collapse automatically. The router is a tool used to cut a printed circuit board (PCB). A few types of defects could occur in the routers and cause unexpected damage to the PCBs. Among these defects, collapse is the most critical defect that must be detected. Currently, router manufacturing companies rely on human inspectors to control the router quality. We first extract the silhouette edges and associated features (peaks and valleys) of a router’s silhouette image by computer vision technique. Then, these silhouette edges and associated features are used to reconstruct a set of 2D isograms that correspond to the router surface. Finally, a pattern recognition method is devised to identify and classify some features of the pattern in the 2D isograms. In this study, two types of routers with different diameters are used for inspection experiments. There are 15 routers of each type. The experimental results reveal that the proposed AOI system can robustly and successfully detect the collapse of diamond-patterned routers with different sizes. The successful detection rate is above 96%. The proposed AOI system can assist in determining the quality of the routers.     

Large scale image annotation: learning to rank with joint word-image embeddings

Image annotation datasets are becoming larger and larger, with tens of millions of images and tens of thousands of possible annotations. We propose a strongly performing method that scales to such datasets by simultaneously learning to optimize precision at k of the ranked list of annotations for a given image and learning a low-dimensional joint embedding space for both images and annotations. Our method both outperforms several baseline methods and, in comparison to them, is faster and consumes less memory. We also demonstrate how our method learns an interpretable model, where annotations with alternate spellings or even languages are close in the embedding space. Hence, even when our model does not predict the exact annotation given by a human labeler, it often predicts similar annotations, a fact that we try to quantify by measuring the newly introduced “sibling” precision metric, where our method also obtains excellent results.     

Affects on visual search performance of individual differences in fixation time and number of fixations

Abstract

The purpose of this study was to provide a basis for improving individual visual performance of inspectors. The relationship between the correct count rate and eye movements of subjects when they counted dots arranged on samples presented for different lengths of time were analysed mainly to determine individual differences. Subjects' eye movements were measured with a corneal reflectance eye camera and analysed frame by frame with a video motion analyser. It was found that accuracy of visual inspection does not depend on length of search time and that a fast search time is not incompatible with a slow search speed. Furthermore, fixation time and number of fixations were considered the main factors governing accuracy of visual inspection. When limited time is allowed for search, a search strategy of prolonging the fixation time leads to high performance and consequently shorter inspection time. Several other findings were obtained which appear important in obtaining accurate information rapidly.     

A feasibility study of ic chip wire bond inspection

Product inspection, which is an important step in industrial manufacturing, has depended on human inspectors whose performance is generally inadequate and variable. In the past few years, automated machines for wire bonding have been developed and integrated in the production of quality IC chips for the electronic industry. However, inspection of the wire bonds has not been automated as yet. This article describes a feasibility study for the automated visual inspection of IC chip wire bonds conducted in the Computer Vision Research Laboratory at Arizona State University. Both the system setup and algorithms for detecting some of the typical wire bonding defects such as ball bond, stitch bond placement, and size defects, and connecting wire defects such as missing and shorted wires using two-dimensional digital images are described.     

Distributed, collaborative software inspection

The Collaborative Software Inspection (CSI) tool, which provides a distributed, structured environment for performing inspections on all software-development products, including specifications, designs, code, and test cases, is described. The inspection environment lets geographically distributed inspection participants meet with people in other cities through workstations at their desks. The current version of all material is accessible online. Inspection products are created online, so secondary data entry to permanent records is not necessary. The inspection information is also available for review and metrics collection. To assess the effectiveness of inspection in the distributive collaborative environment and compare it with face-to-face meetings, a case-study approach with replication logic is presented     

Multipurpose inspection task simulator

Experimentation using a computer simulation of a system has many advantages over more traditional techniques. In this project, computer graphics are used to create an inspection task simulator. The simulator allows for easy manipulation of variables known to affect inspection performance, quick construction of visual stimuli, a choice of randomly generated or intentionally input faults on the stimuli, controlled % defective, feedback options and automatic collection of inspection data. The inspection task simulator has possible applications in experimentation, training and selection of inspectors.