Evaluating requirements modeling methods based on user perceptions: A family of experiments

Numerous methods and techniques have been proposed for requirements modeling, although very few have had widespread use in practice. One drawback of requirements modeling methods is that they lack proper empirical evaluations. This means that there is a need for evaluation methods that consider both the theoretical and practical aspects of this type of methods and techniques. In this paper, we present a method for evaluating the quality of requirements modeling methods based on user perceptions. The evaluation method consists of a theoretical model that explains the relevant dimensions of quality for requirements modeling methods, along with a practical instrument with which to measure these quality dimensions. Basically, it allows us to predict the acceptance of a particular requirements modeling method in practice, based on the effort of applying the method, the quality of the requirements artifacts produced, and the user perceptions with regard to the quality of the method. The paper also presents an empirical test of the proposed method for evaluating a Rational Unified Process (RUP) extension for requirements modeling. That test was carried out through a family of experiments conducted with students and practitioners and provides evidence of the usefulness of the evaluation method proposed.     

基于用户角度的网站服务质量评价研究

用户对一个网站的喜爱,是建立在网站服务质量上的,这也是网站设计的追求目标。文章以SERVQUAL和TAM模型为基础,讨论从用户角度如何获得满意的服务质量,赢得用户的认可。用户对网站的期望和认可,是服务质量评价的关键因素。从三个方面阐述:对网站设计的感知,服务质量评价的维度和网站质量的评价结果。     

Development and usability testing of an operating manual for a hair‐braiding machine

The present study applied the concepts and techniques of usability testing to the development of an operating manual for a hair‐braiding machine. A preliminary version of the manual was developed in four stages: (1) profiling user characteristics, (2) defining design features of the manual, (3) preparing contents, and (4) designing the format and layout of the manual. In each development stage, design factors, such as user characteristics and operating instructions, were identified and relevant methods, such as user profile analysis and hierarchical task analysis, were applied. The preliminary manual was evaluated by 5 cosmetology students (age, 18 to 25 years; training, 60 to 700 hours) by using various performance and preference measures and surveying opinions of the participants regarding likes, dislikes, and suggestions. Based on the test results, recommendations were made for better usability of the manual and revisions were made accordingly. The design and evaluation process of an operating manual presented in the study would help practitioners to develop a user‐friendly manual. © 2007 Wiley Periodicals, Inc.     

渐进式虹膜图像质量评估模型

虹膜图像质量评估目前尚无统一评估标准,导致虹膜识别拒识率和误识率较高。针对该问题,提出一种虹膜图像质量评估模型。根据虹膜图像中各干扰因素的不同特点,在先验知识基础上采用区域化、加权的方法,渐近式地实施像素级质量评估,依据像素级评估结果实施图像级质量评估。实验结果表明,像素级虹膜图像质量评估中的虹膜图像干扰项识别率和模糊识别率较高,图像级虹膜图像质量评估与人工评估结果相一致。     

The quality of user documentation

Encouraging quality user documentation is an important area for top management involvement in MIS development. User documentation enhances the value of an application to the user and improves user satisfaction. Yet, no practical managerial tools for measuring the quality of user documentation exist. This paper develops two instruments for measuring user perceptions of the quality of their documentation. The reliability and validity of these instruments are examined. Where the goal is to identify specific strengths and weaknesses of existing user documentation, a more comprehensive fifteen item instrument is recommended. Where only a summary evaluation is desired, a short 3 item alternative form proves sufficient. With these instruments, management can more easily monitor and control the condition of their firm's user documentation.     

软件质量评价体系及其实现

近年来,随着国际竞争的不断升级,“质量是软件的生命”这一口号已深入人心。软件质量评价作为软件质量保证的重要手段,已逐渐被开发商、用户所重视。世界上有许多机构开始进行质量评价的形式化手段研究,并推出了如DIN66285,ISO9126等软件质量评价标准。本文就是对这一领域的研究,主要分析了软件质量的科学评价方法,总结了得出的一些算法与心得,在国内首次提出了评价模型分级的设想,并在计算机上加以实现。     

A study of user acceptance tests

The user acceptance test (UAT) is the final stage of testing in application software development. When testing results meet the acceptance criteria, the software system can be released for operational use.This paper first compares the different testing phases of software development (i.e. unit test, integration test, system test and UAT) in terms of important testing elements so as to highlight the uniqueness of UAT relative to the other test phases. Then, we describe several approaches for acceptance test, including the behaviour-based approach, the black-box strategy and a new UAT strategy called operation-based testing. The new strategy uses the operational profile for testing purposes, includes a well-defined acceptance criteria, and satisfies the test requirements of ISO9001 standards.     

Quality metrics for intranet applications

As the number of intranet application increases, software developers face a new software paradigm and possibly a new set of quality requirement. The work discussed here attempts to identify practical software metrics for intranet applications. The six software quality characteristics and 32 quality sub-characteristics of the Extended ISO model are used as a basis to identify the key quality characteristics of intranet applications. From the results of a user survey, three key quality characteristics are identified; they are reliability, functionality, and efficiency. Five sub-characteristics (availability, accuracy, security, suitability and time behaviour) are found to be the key attributes of intranet applications. Finally, a set of three metrics is developed. In order to verify their validity and applicability to intranet projects, an experiment was performed by computing these metrics in five intranet applications. The metric values were then compared with the results from a user satisfaction survey. The metric values and the survey results were closely correlated; a larger score in software metrics leads to a higher user satisfaction score. It is very likely that these software metrics can effectively reflect the quality of an intranet application.     

Usability measurement and metrics: A consolidated model

Usability is increasingly recognized as an important quality factor for interactive software systems, including traditional GUIs-style applications, Web sites, and the large variety of mobile and PDA interactive services. Unusable user interfaces are probably the single largest reasons why encompassing interactive systems – computers plus people, fail in actual use. The design of this diversity of applications so that they actually achieve their intended purposes in term of ease of use is not an easy task. Although there are many individual methods for evaluating usability; they are not well integrated into a single conceptual framework that facilitate their usage by developers who are not trained in the filed of HCI. This is true in part because there are now several different standards (e.g., ISO 9241, ISO/IEC 9126, IEEE Std.610.12) or conceptual models (e.g., Metrics for Usability Standards in Computing [MUSiC]) for usability, and not all of these standards or models describe the same operational definitions and measures. This paper first reviews existing usability standards and models while highlighted the limitations and complementarities of the various standards. It then explains how these various models can be unified into a single consolidated, hierarchical model of usability measurement. This consolidated model is called Quality in Use Integrated Measurement (QUIM). Included in the QUIM model are 10 factors each of which corresponds to a specific facet of usability that is identified in an existing standard or model. These 10 factors are decomposed into a total of 26 sub-factors or measurable criteria that are furtherdecomposed into 127 specific metrics. The paper explains also how a consolidated model, such as QUIM, can help in developing a usability measurement theory. Ahmed Seffah interests are at the intersection of human-computer interaction and software engineering, with an emphasis on human-centered software engineering, empirical studies, theoretical models for quality in use measurement, as well as patterns as a vehicle for capturing and incorporating empirically valid design practices in software engineering practices. He is a co-founder of the Usability and Empirical Studies Lab and the founder and the chair of the Human-Centered Software Engineering Research Group at Concordia University. Harkirat K. Padda is a Ph.D. candidate at Concordia University (Montreal) since 2003, where she completed her masters’ degree in computer science. Her research interests are software quality measurement, metrics, and empirical software evaluation. Being a member of Human Computer Software Engineering Group, she is exploring the comprehension and usability of software systems—in particular the visualization systems. In her masters’ work, she already defined a repository of different metrics to measure the ‘quality in use’ factors of software products in general. Currently, she is proposing a pattern-oriented measurement framework to measure comprehension of visualization systems.     

Weight-modeling of B2C system quality

The efficiency and cost effectiveness of internet technologies have already transformed the web into a global environment for business. However, designing, developing and supporting e-commerce systems having quality in mind is a challenging task. The quality of a Business to Consumer system may be assessed from two complementary, orthogonal, points of view: as a software system and as a service to end-users/customers. As a software system it must be assessed by professional software engineers, the evaluators. They are best in assessing top down the system. As a service it must be assessed by customer's perspective, the end-user. They provide a bottom up evaluation approach of the system. In this work, these points of view are combined in a weighted model which uses the external quality characteristics and sub-characteristics of ISO9126 as a baseline for further decomposition into technical and user-oriented features. The model can be used forward and backwards: forward for evaluating in detail the qualitative strengths and weaknesses of an existing B2C system; backwards for balancing quality improvement with development criteria.     

E-commerce system quality assessment using a model based on ISO 9126 and Belief Networks

As business transitions into the new economy, e-system successful use has become a strategic goal. Especially in business to consumer (e-commerce) applications, users highly evaluate the quality of their interactive shopping experience. However, quality is difficult to define and measure and most importantly, it is difficult to measure its impact on the end-user. Among the many research questions that arise, some of the most important concern the exact nature of the quality attributes that define an e-commerce system, and how one could model these attributes in order to increase its acceptance. Bearing in mind that e-commerce systems are actually user/data-intensive web-based software systems, this work performed a survey which resulted in a theoretical model that helps to measure such systems’ dynamics through their decomposition into primary quality characteristics. The proposed model is based on Bayesian Networks and ISO 9126. Besides the emphasis on specific software quality attributes, it also provides a quality assessment process aiding developers to design and produce e-commerce systems of high quality. Using a Bayesian Network the model can be used to combine different types of evidences and provide reasoning from effect to cause and vice versa.

The impact of Web quality and playfulness on user acceptance of online retailing

We investigated the effect of playfulness on user acceptance of online retailing and tested the relationship between Web quality factors and user acceptance behavior. A survey of 942 users of Web-based online retailing was conducted to test our model.     

基于图像评价的灯箱最佳成像环境因素确定方法*

为获得高质量的水稻叶部病害图像,在自行设计的灯箱中充分考虑了物距、灯管数目、灯管高度、灯管侧距等成像环境因素对图像质量的影响,通过主客观评价方法对不同成像环境因素水平下拍摄的照片进行质量评价,最终得出最佳成像环境因素取值水平。该过程分为三步：首先采用正交试验法设计各环境因素取值水平并拍摄照片,然后采用客观评价方法进行图像质量评价,最后通过与主观评价方法对比确定成像环境因素最佳取值水平。经过大量实验分析得出：在不同成像环境因素取值水平下,所拍摄照片的质量是有差别的。因此,基于质量评价的方法来确定最佳成像环境     

Extending the engineering trade-off analysis by integrating user preferences in conjoint analysis

The ongoing technical improvements in architecture design with improved features of mobile or smartphones do not automatically guarantee user acceptance, because technical and commercial aspects primarily drive the development of mobile communication systems and devices. Especially in early stages of technology development, user preferences and values are not adequately considered, which might even have a negative impact on acceptance issues. The aim of this study was the implementation of a quantified understanding of user needs in terms of values into the system design process of cell-phone processors. Moreover, we aimed for an extension of the engineering’s trade-off analysis by using conjoint analysis in order to investigate trade-offs between specific device characteristics. Finally, our aim was the evaluation of empirically based user-oriented research methods.Results of the first study revealed that battery life, speech quality, signal quality and data-transmission rate are the most important device characteristics. Results from conjoint analysis indicated a clear trade-off between battery life and the three other characteristics. Moreover, this research demonstrated that technology acceptance research benefits considerably from an interdisciplinary and multi-method approach. Besides, implementing the users’ preferences into early stages of the product development process offers several advantages concerning effectiveness as well as economic aspects of development.