Test and evaluation of distributed information system networks

As information services become increasingly ubiquitous, and are being charged for, users are demanding the ability to choose the quality of an information service based on its cost. In this environment, delivering the right information to the right user at the right time, and with appropriate quality, is the challenge. In addition, resource optimizations must be performed to the degree possible. In this paper we introduce the concept of information quality based system evaluation. The key contribution is a framework in which user/application needs for information quality can be quantified in terms of rigorously defined metrics, existing systems can be evaluated against these metrics, and the tradeoffs between information quality and its cost of provisioning can be examined. Further, this provides an approach to developing information quality aware policies and mechanisms for the system. This framework has been partially validated by applying it to the provisioning of continuous media services, i.e., audio and video, in a distributed environment. This paper presents the case study in detail. Finally, the paper outlines a research program in the area of information quality based system evaluation. This revised version was published online in June 2006 with corrections to the Cover Date.