Planning and Implementing a Federated Searching System: An Examination of the Crucial Roles of Technical,Functional, and Usability Testing

SUMMARY

Federated searching software offers much promise to users as a convenient way of accessing the wealth of electronic information resources libraries provide. But metasearching is not the same as Google searching; care must be taken in organizing and presenting search options and results so that they are comprehensible to users. From software installation to usability testing and creating documentation, most of the work of implementation is behind-the-scenes and hidden from most library staff and users; however, the decisions made during implementation greatly affect staff and user experiences with the product as well as its overall utility and usability. Systematic testing of the product is necessary to make informed and defensible decisions. This article details three layers of testing (technical, functional, and usability) recommended during implementation of a federated search product, based on best practices in the literature, metasearch standards, and the authors' own experiences with implementing a locally developed broadcast search system and the federated search system WebFeat.     

Translating Technical Data into Effective User Manuals

Abstract

Every software system, whether designed for commercial or noncommercial use, must be documented. Documentation must be geared to users' needs and knowledge; it should be neither too technical nor too simple. The quality of documentation is a major determinant of how well a system is received and how widely it is used.     

A NEW LOOK AT DOCUMENTATION AND TRAINING

Users need quality documentation and training. A technical communicator can be an indispensable communications analyst and problem solver for an organization, helping to analyze user requirements, clarify the documentation process, develop information products, and provide computer training. By assisting IS management in solving increasingly complex communication problems, this person can also shed the one-dimensional image of the technical writer who works solely as a software documenter at the end of the development process.     

Human factors in user documentation

Writing user documentation is a job most programmers do not enjoy and are not normally trained in. As computer users become aware of the importance of ease of use, so must computer suppliers be aware of ways to improve user documentation. As well as abilities in technical communication, people who generate high quality computer user documentation require system programming experience and human understanding of computer users. The paper relates the importance of good documentation to user productivity, analyzes the process of generating documentation, and discusses the skills needed by persons employed in that industry.     

How are software defects found? The role of implicit defect detection,individual responsibility,documents, and knowledge

ContextPrior research has focused heavily on explicit defect detection, such as formal testing and reviews. However, in reality, humans find software defects in various activities. Implicit defect detection activities, such as preparing a product demonstration or updating a user manual, are not designed for defect detection, yet through such activities defects are discovered. In addition, the type of documentation, and knowledge used, in defect detection is diverse.ObjectiveTo understand how defect detection is affected by the perspectives of responsibility, activity, knowledge, and document use. To provide illustrative numbers concerning the multidimensionality of defect detection in an industrial context.MethodThe data were collected with a survey on four software development organizations in three different companies. We designed the survey based on our prior extensive work with these companies.ResultsWe found that among our subjects (n = 105), implicit defect detection made a higher contribution than explicit defect detection in terms of found defects, 62% vs. 38%. We show that defect detection was performed by subjects in various roles supporting the earlier reports of testing being a cross-cutting activity in software development organizations. We found a low use of test cases (18%), but a high use of other documents in software defect detection, and furthermore, we found that personal knowledge was applied as an oracle in defect detection much more often than documented oracles. Finally, we recognize that contextual factors largely affect the transferability of our results, and we provide elaborate discussion about the most important contextual factors. Furthermore, we must be cautious as the results were obtained with a survey, and come from a small number of organizations.ConclusionsIn this paper, we show the large impact of implicit defect detection activities in four case organizations. Implicit defect detection has a large contribution to defect detection in practice, and can be viewed as an extremely low-cost way of detecting defects. Thus, harnessing and supporting it better may increase quality without increasing costs. For example, if an employee can update the user manual, and simultaneously detect defects from the software, then the defect detection part of this activity can be seen as cost-free. Additionally, further research is needed on how diverse types of useful documentation and knowledge can be utilized in defect detection.     

Architectural knowledge discovery with latent semantic analysis: Constructing a reading guide for software product audits

Architectural knowledge is reflected in various artifacts of a software product. In a software product audit this architectural knowledge needs to be uncovered and its effects assessed in order to evaluate the quality of the software product. A particular problem is to find and comprehend the architectural knowledge that resides in the software product documentation. In this article, we discuss how the use of a technique called Latent Semantic Analysis can guide auditors through the documentation to the architectural knowledge they need. We validate the use of Latent Semantic Analysis for discovering architectural knowledge by comparing the resulting vector-space model with the mental model of documentation that auditors possess.     

An interactive support tool for writing multilingual manuals

Machine translation has been the dominant paradigm for automated multilingual document production. In this paradigm, a technical writer generates a source text, which is translated by the computer system into another language and then edited. One problem with machine translation, however, is that its output is typically constrained by the original text's style and language. Automatic language-generation systems, however, start with an underlying knowledge base that represents the text's content without dictating its language or style. However, most automatic systems are stand-alone tools, leaving technical writers out of the loop. The systems assume that an underlying knowledge base containing the necessary information is available or can be easily obtained. This is not necessarily the case, though, when producing instruction manuals. For example, the knowledge base required to produce instructions should contain user-oriented information. User-oriented documentation, which concerns the ways the product can help users achieve their goals, is more effective than documentation that focuses on the product. Only a technical writer can specify user-oriented information. It is thus prefer able to have a document-generation system that works with the writer. With this in mind, we developed Drafter, an interactive document drafting tool that can be integrated into the technical writers' working practices and that can automatically and simultaneously generate appropriately worded drafts in several languages. Drafter's current domain of application is software manuals     

Knowledge-based approaches in software documentation: A systematic literature review

ContextSoftware documents are core artifacts produced and consumed in documentation activity in the software lifecycle. Meanwhile, knowledge-based approaches have been extensively used in software development for decades, however, the software engineering community lacks a comprehensive understanding on how knowledge-based approaches are used in software documentation, especially documentation of software architecture design.ObjectiveThe objective of this work is to explore how knowledge-based approaches are employed in software documentation, their influences to the quality of software documentation, and the costs and benefits of using these approaches.MethodWe use a systematic literature review method to identify the primary studies on knowledge-based approaches in software documentation, following a pre-defined review protocol.ResultsSixty studies are finally selected, in which twelve quality attributes of software documents, four cost categories, and nine benefit categories of using knowledge-based approaches in software documentation are identified. Architecture understanding is the top benefit of using knowledge-based approaches in software documentation. The cost of retrieving information from documents is the major concern when using knowledge-based approaches in software documentation.ConclusionsThe findings of this review suggest several future research directions that are critical and promising but underexplored in current research and practice: (1) there is a need to use knowledge-based approaches to improve the quality attributes of software documents that receive less attention, especially credibility, conciseness, and unambiguity; (2) using knowledge-based approaches with the knowledge content in software documents which gets less attention in current applications of knowledge-based approaches in software documentation, to further improve the practice of software documentation activity; (3) putting more focus on the application of software documents using the knowledge-based approaches (knowledge reuse, retrieval, reasoning, and sharing) in order to make the most use of software documents; and (4) evaluating the costs and benefits of using knowledge-based approaches in software documentation qualitatively and quantitatively.     

A Multi-Window Hypermedia Environment for the Delivery of Information to Architects,Engineers and Product Designers

A report is given of the Quality Package: a multi-window hypermedia system delivering information on Design Quality to architects, engineers, and product designers. An innovative information processing structure was devised including an object-oriented communication protocol, and a composite user interface individually tailored to individual needs.

The information content is described to identify the manner in which modes of interaction were reflected in the separation of content accessed by menu windows. The flow of information was maintained through the use of link commentaries selected according to tests of user performance. Further interactions were also possible through agree and disagree decisions aggregated in a performance statement in the package summary. The user could therefore use their own interactions to assess personal notions of design quality in comparison with statements from established experts personified in videodisc and compact audiodisc material.

Conclusions are drawn concerning the applicability of the further development of the multiwindow communication philosophy to create flexible modes of information delivery.     

Don't underestimate the problems of user centredness in software development projectsthere are many!

Abstract

On the basis of a longitudinal field study of 29 commercial software development projects, the pros and cons of user centredness in software development were analysed. We looked at two concepts: user participation-an organizational device-involving a user representative in the team, and user orientation-a cognitive-emotional concept-which pertains to positive attitudes towards users. Both were found to be associated with project difficulties relating to process and product quality as well as overall project success. We suggest that the issue is no longer whether or not to involve users, but instead to develop a realistic understanding of the difficulties associated with user centredness.     

Organizing human factors knowledge for the evaluation and design of interfaces

This paper presents research conducted in the area of human factors knowledge collection and organization. Providing that a large amountof human factors engineering data is available now concerning user interface aspects, this research started from the following question: Can this knowledge be transferred to user interface designers, and by which means can this be done? An important part of human factors knowledge is included in recommendations and guidelines. It is the problems with that type of knowledge that this paper focuses on. A first attempt to tackle some of these problems is made by offering a generic model for deciphering recommendations into rules. This model stresses the importance of attributes such as human factors criteria, classes of premises, and conclusions, as well as interface objects. Definitions of criteria for organizing recommendations are offered, and a few examples of rule‐based reasoning are provided. The remaining problems with a rule approach are then discussed according to two dimensions: intrinsic problems and usage problems. In addition, it appears that an approach based on literature recommendations is not sufficient. Other areas that need further work are described, particularly concerning task and interface modelling, and human factors expertise modelling. The paper concludes on the limits and benefits of such an approach.     

LIY: learn-it-yourself software interfaces

An increasing number of people are becoming users of unfamiliar software. They can be genuinely "new" computer users or part of a growing group who are transferring skills and knowledge from a familiar product such as a word processor to a functionally similar, but different, unfamiliar one. The problem for users in this position is that they do not have access to training courses to teach them how to use such software and are usually forced to rely on text-based documentation. LIY is a method for producing computer-based tutorials to teach the user ofasoftware product.This paper describes how LIY is, in turn, (1) a method for application system design which recognizes the need for tutorial design (a task analysis and user interface specification provide information structures that are passed to the tutorial designer); (2) a support environment for the tutorial designer (in addition to prompting for courseware for nodes in the task analysis, LIY provides a ready-made rule base for constraining the degree of learner control available while the tutorial is in use. The designer is able to tailor this rule base for a specific tutorial); and (3) a tutorial delivery environment (the tutorial adapts to individual learners and offers a degree of learner control).     

Cima: An interactive concept learning system for end-user applications

Personalizable software agents will learn new tasks from their users. In many cases the most appropriate way for users to teach is to demonstrate examples. Learning complex concepts from examples alone is hard, but agents can exploit other forms of instruction that users might give, ranging from yes/no responses to ambiguous, incomplete hints. Agents can also exploit background knowledge customized for applications such as drawing, word processing, and form filling. The Cima system learns generalized rules for classifying, generating, and modifying data, given examples, hints, and background knowledge. It copes with the ambiguity of user instructions by combining evidence from these sources. A dynamic bias manager generates candidate features (attribute values, functions, or relations) from which the learning algorithm selects relevant ones and forms appropriate rules. When tested on dialogs observed in a prior user study on a simulated interface agent, the system achieved 95% of the learning efficiency observed in that study.     

Instructions and demonstration as media for training new users of Automatic Speech Recognition Devices

Abstract

Although automatic speech recognition (ASR) can provide a medium of controlling computers which is relatively easy to use, novice users often have problems with it during their initial practices. In this study, two methods for training subjects to use ASR are compared. One group of subjects received a short demonstration given by an experienced ASR user and the other group received verbal instructions on how to use the device. The results show that subjects given a demonstration achieved better performance than those given instructions (p< 0.005). This is explained by virtue of the fact that the successful use of ASR requires procedural knowledge which is better acquired through some form of practice than through instruction. It is concluded that a demonstration provides ‘practice by proxy’. ‘Task like’ forms of enrolment are discussed. It is suggested that although they can provide the possibility of practice, they are not applicable to all types of ASR use. A demonstration provides users with task familiarization, and an appropriate style of speech.     

Human factors-pruning your program's unused functions

The author cites a common finding of human factors research on software that about 10% of the functions account for almost 90% of the product's use. She discusses the negative effect of having many features that are seldom used, namely: the product is (usually) more expensive; programs require more memory; the product is harder to use; the documentation is more bulky, which makes it more difficult to use; and the user interface may not be optimally designed. For software developers who want to optimize their programs by getting rid of useless features she describes three ways to find out what the users really need and actually use: task analysis, data logging, and surveys and interviews     

Evaluation of user interfaces: EVADIS II— a comprehensive evaluation approach

Abstract

As a result of the importance of the usability approach in system development and the EC's ‘Directive concerning the minimum safety and health requirements for VDT workers’ (EWG 1990), there is an accepted need for practical evaluation methods for user interfaces. The usability approach and the EC Directive are not restricted to user interface design, as they include the design of appropriate hardware and software, as well as organization, job, and task design. Therefore system designers are faced with many, often conflicting, requirements and need to address the question, ‘How can usability requirements comprehensively be considered and evaluated in system development?’ Customers buying hardware and software and introducing them into their organization ask, (How can I select easy-to-use hardware and software?’ Both designers and customers need an evaluation procedure that covers all the organizational, user, hard- and software requirements. The evaluation method, EVADIS.II, we present in this paper overcomes characteristic deficiencies of previous evaluation methods. In particular, it takes the tasks, the user, and the organizational context into consideration during the evaluation process, and provides computer support for the use of the evaluation procedure.     

An efficient approach to understanding and predicting the effects of multiple task characteristics on performance

Abstract

In ergonomics there is often a need to identify and predict the separate effects of multiple factors on performance. A cost-effective fractional factorial approach to understanding the relationship between task characteristics and task performance is presented. The method has been shown to provide sufficient independent variability to reveal and predict the effects of task characteristics on performance in two domains. The five steps outlined are: selection of performance measure, task characteristic identification, task design for user trials, data collection, regression model development and task characteristic analysis. The approach can be used for furthering knowledge of task performance, theoretical understanding, experimental control and prediction of task performance.

Practitioner Summary: A cost-effective method to identify and predict the separate effects of multiple factors on performance is presented. The five steps allow a better understanding of task factors during the design process.     

Effects of time affordance and operation mode on a smart microwave oven touch-sensitive user Interface design

Smart homes have revolutionized our daily lives. With today’s fast-paced lifestyle, pursuing a high quality of life has become many people’s goal and motivation. The purpose of this study is to investigate the user interface design of smart microwave ovens pertinent to time affordance and operation mode. A 2?×?3 mixed factorial design was planned to help explore whether different time affordances (i.e., high and low) and operation modes (i.e., traditional, touch, and smart) may affect users’ task performance and subjective evaluation. Using the convenience sampling method, 24 adults were recruited to participate in the experiment. The experimental data were collected pertinent to task performance, the system usability scale, and through questionnaires created using a 7-point Likert scale, and semi-structured interviews. The generated results revealed that: (1) there was a significant difference in the time affordance. Multiple time information cues can help reduce uncertainty, providing high time affordance to the participants. (2) There were significant differences among different operation modes. A simple and intuitive “smart” type is in line with user expectations. (3) The overall analysis of task performance and satisfaction consistently showed that high time affordance is better than low time affordance in all aspects, and the “smart” type had the best task performance. (4) The user interface design should be followed by users’ experience and the features of the touch product. Partially smart and custom function adjustments may effectively improve the user’s control of smart products.