Does Every Inspection Really Need a Meeting?

Software review is a fundamental component of the software quality assurance process, yet significant controversies surround the most efficient and effective review method. A central question surrounds the use of meetings; traditional review practice views them as essential, while more recent findings question their utility. To provide insight into this question, we conducted a controlled experiment to assess several measures of cost and effectiveness for a meeting and non-meeting-based review method. The experiment used CSRS, a computer mediated collaborative software review environment, and 24 three person groups. We found that the meeting-based review method studied was significantly more costly than the non-meeting-based method, but that meeting-based review did not find significantly more defects than the non-meeting-based method. However, the meeting-based review method was significantly better at reducing the level of false positives, and subjects subjectively preferred meeting-based review over non-meeting-based review. This paper presents the motivation for this experiment, its design and implementation, our empirical findings, pointers to Internet repositories for replication or additional analysis of this experiment, conclusions, and future directions.