User Expectations in the Time of Google: Usability Testing of Federated Searching

SUMMARY

As libraries expand their services into the world of federated searching, librarians need to work with users to discover what their expectations are and how the library can customize the software to meet users' expectations. This article describes the user testing performed at Texas A&M University during 2005 as the libraries implemented a new federated search service called Search Now (ExLibris' MetaLib). Over fifty volunteers–including undergraduates, graduate students, faculty, and library faculty and staff–helped to test the new system and offered suggestions for improvements. Problems were noted and, where possible, modifications were made to improve results. These modifications were then tested again. Major issues noted during the usability testing included: user expectations of search performance; information included in and the layout of the search results; availability of advanced search options; and lack of ability to limit by format, scholarly nature of journal, date and full-text availability. Suggestions for further development are also presented.     

Continuing the Quest for the Quick Search Holy Grail

ABSTRACT

This article describes the selection, implementation, configuration, and assessment of a proprietary federated search product. The decisions made and processes used to facilitate the implementation are outlined. Users' perception of the product based on a usability study and survey responses are presented. Also described are unique and innovative implementations of the software within Oregon State University libraries and the application of this experience to the creation of a homegrown open source federated search tool.     

A Review of “I Found it on the Internet: Coming of Age Online”

SUMMARY

This article will address the problems of developing and implementing a federated searching system and the involvement by reference staff expected to use, teach, and promote it. Implementation of federated search usually begins with developing basic technical functionality and then progresses to increased participation by public service librarians and system users in order to refine searching interfaces and content retrieval mechanisms and display. From the reference perspective, some impediments occurred in utilizing the federated search system since reference staff did not participate in the development of the product; since there was resistance to the concept of the product by instructional staff; and since performance inconsistencies and the use of a complex interface hampered functionality. This article will address the evolution of organizational stakeholder participation in the development of the federated search system at the Loyola Notre Dame Library.     

Preparing Staff for Federated Searching: A Community of Practice Approach

ABSTRACT

This article outlines one library's staff-training program for a newly implemented federated search product and lessons learned from the process. The discussion examines federated searching from the patron's point of view and introduces a community of practice approach to training staff. The approach aims to help staff learn how to embrace federated searching as an additional research tool to complement search features already in place.     

Protocol Analysis of a Federated Search Tool: Designing for Users

SUMMARY

Librarians at Springfield College conducted usability testing of Endeavor's federated search tool, ENCompass for Resource Access. The purpose of the testing was to make informed decisions prior to customizing the look and function of the software's interface in order to make the product more usable for their patrons. Protocol, or think-aloud, analysis was selected as a testing and analysis method. Subjects from the general college community were recruited and given a list of tasks to perform on ENCompass, and they were asked to speak all of their thoughts out loud as they worked. Upon analyzing the test results, researchers found that subjects' problems fell into certain categories, such as unfamiliarity with terms or navigation from screen to screen. The researchers were able to use their findings to recommend extensive revisions to the interface, which improved usability for this library's patrons.     

Federated Searching

ABSTRACT

With an increasing number of libraries implementing federated searching to help users negotiate the complex electronic database environment, it is important to note how users are being instructed to use these resources. This article presents the results of examinations of the Web sites of Association of Research Libraries (ARL) member institutions that currently use federated search products. It analyzes help guides and search tips related to federated searching to determine what concepts are being explained to users and how the information is being communicated. The result of the evaluation of these Web sites is a list of “best practice” guidelines that may be used by librarians creating their own federated searching guides. doi:10.1300/J136v12n03_03     

Build It (and Customize and Market It) and They Will Come

ABSTRACT

Like other academic libraries, the University Library System of the University of Pittsburgh (ULS) continues to invest in providing anytime, anywhere access to research materials in electronic format. While there are obvious benefits for users in this concomitant increase in complexity for these same users, with many of them, especially undergraduates, turning to the speed and simplicity of Google and away from the complexity of library sponsored electronic resources. Seeking ways to both maximize our investment in full text resources, and promote easy access to this universe of electronic resources for our users, the ULS felt strongly that a navigational solution had to be found, and in 2004 began the process of implementing a federated search tool.

This article discusses the collaborative process between the University Library System and its federated search vendor, and discusses the decision making process behind the customizations made by the ULS to its federated search system. The article also addresses the process of working with a professional marketing organization to developing a marketing message and supporting materials to promote its federated search system, and finally briefly looks at usability and usage statistics to assess the degree to which the Zoom! federated search system has met the goal of providing fast, easy access to high quality information. doi:10.1300/J136v12n03_07     

Challenges for Federated Searching

ABSTRACT

This article looks at the current capabilities of federated search software applications which search multiple full text e-journal databases. It also considers the developing capability to search journal content over the World Wide Web. Advances in federated searching are considered, such as standardized XML based searching, subject subdivided searching, linking services and Web indexing of formerly proprietary content. Despite these advances, federated searching of e-content so far falls short of its potential and of user expectations. The article looks at developments which are still needed to create a unified cross-searchable electronic content environment out of the current collection of separate e-content sources. It explores a number of challenges which must be overcome by e-content and software vendors, network developers and libraries to allow federated search and access to rise to a new level well beyond current limited capabilities. doi:10.1300/J136v12n03_08     

The Accuracy and Thoroughness of a Federated Search Engine in the Health Sciences

SUMMARY

Research studies show that researchers need to access multiple resources to ensure a comprehensive search. In the health care field, health professionals' time is very limited, and they would benefit from being able to access multiple resources simultaneously. Health professionals must retrieve the most relevant and accurate information possible when applying clinical information to patient care. While a federated search engine may offer speedy search capabilities, do they offer the same accuracy and relevancy as directly searching the core health sciences databases? The purpose of this study is to describe, evaluate, and compare searching health sciences topics using a federated search engine versus searching five core health sciences databases directly. The following databases were selected for this study: WebFeat (federated search engine), MEDLINE, CINAHL, Science Citation Index, Biological Abstracts, and EMBASE. While WebFeat is convenient for searching multiple databases simultaneously, it does not offer the advanced search limits offered by directly searching some of the health sciences databases. The lack of limits available in the federated search engine means increased results that may not be relevant to the busy health care professional.     

Initiating the Learning Process

ABSTRACT

Access to federated search tools is increasing and many academic libraries are now looking beyond implementation, and are considering the broader implications of federated searching such as its impact on information literacy programs. Although this emerging technology is still experiencing acknowledged growing pains, its potential for transforming the way students conduct research is promising. Reviewing information seeking behavior research and analyzing how federated searching fits within the Association of College and Research Libraries' Information Literacy Competency Standards for Higher Education (2000) enables academic librarians to consider an integrative instructional model that utilizes federated searching as a way of initiating the learning process. doi:10.1300/J136v12n03_01     

A Nontraditional Application of MetaSearch

ABSTRACT

Federated searching has become an increasingly popular way for academic libraries to provide access to their diverse and rapidly growing collections of electronic resources. IUPUI University Library handled the implementation of its federated search project somewhat differently than many academic libraries. The cornerstone of their efforts was to not only create a gateway to the library's electronic resources, but to build upon previous experience as a beta site for Ex Libris' SFX cross-referencing software by using MetaSearch in conjunction with SFX to permit teaching faculty to link to online bibliographies through the university's course management system, Oncourse. Issues of working with two closed systems needed to be overcome, but a product resulted that was embraced by the faculty and gave direction for future developments. doi:10.1300/J136v12n03_04     

Graduate Students and Federated Searching

In the fall of 2009 and spring of 2010, 20 New Mexico State University graduate students were interviewed using a “think-aloud” protocol to describe their experiences with a federated search tool. Students found federated searching to be most effective when looking for very specific information and article abstracts and least effective when searching general terms and resources other than journal articles (i.e., books, media). Students used the federated search tool and Google together to find relevant citations and full text journal articles. Students indicated that they were highly satisfied with federated search despite some doubt as to how the search worked and what library resources were included in the search.     

The usability engineering life cycle

A practical usability engineering process that can be incorporated into the software product development process to ensure the usability of interactive computer products is presented. It is shown that the most basic elements in the usability engineering model are empirical user testing and prototyping, combined with iterative design. Usability activities are presented for three main phases of a software project: before, during, and after product design and implementation. Some of the recommended methods are not really single steps but should be used throughout the development process     

Tailoring by integration of domain-specific components: the case of a document search tool

In this paper we describe the evolutionary design and implementation of a search tool for files in shared workspaces used within an off-the-shelf groupware product. The design is based on the assumption that a useful generic search tool must be highly tailorable, which is achieved by applying an innovative software architecture allowing the assembly of components during runtime. In order to understand people's searching methods in shared workspaces and to support the design, we employed interviews and workshops with users as well as a field test to understand the users' needs. During the design process we developed a series of prototypes that were then evaluated by office workers. Consequently, the process described and the lessons learned extend from searching in files as a case via tailorability of software as an answer to the resulting requirements to component architecture as a way to implement this tailorability. The results derived from the treatment of these interrelated aspects constitute the core and value of this paper.     

Librarian Perspectives on Teaching Metasearch and Federated Search Technologies

ABSTRACT

This article explores three assessment projects: two focused on librarians' reactions to metasearch technologies from a reference and information literacy perspective, and one user survey that attempts to capture student experiences, understanding, and satisfaction with metasearching at California State University Northridge. Exploring both librarian and user reactions to metasearch and federated search interface technologies in academic libraries, the authors review what the literature reveals about these technologies and their impact on information literacy initiatives. Emphasis focuses on how federated searching technologies affect librarians, students, the marketing of existing library and electronic databases, and information literacy instructional programming. doi:10.1300/J136v12n03_02     

Tailoring by integration of domain-specific components: the case of a document search tool

In this paper we describe the evolutionary design and implementation of a search tool for files in shared workspaces used within an off-the-shelf groupware product. The design is based on the assumption that a useful generic search tool must be highly tailorable, which is achieved by applying an innovative software architecture allowing the assembly of components during runtime. In order to understand people's searching methods in shared workspaces and to support the design, we employed interviews and workshops with users as well as a field test to understand the users' needs. During the design process we developed a series of prototypes that were then evaluated by office workers. Consequently, the process described and the lessons learned extend from searching in files as a case via tailorability of software as an answer to the resulting requirements to component architecture as a way to implement this tailorability. The results derived from the treatment of these interrelated aspects constitute the core and value of this paper.     

User Perceptions of MetaLib Combined Search: An Investigation of How Users Make Sense of Federated Searching

SUMMARY

This article presents an investigation of users' understanding of MetaLib Combined Search (MCS), a federated search system implemented for the Washington Research Library Consortium. Through a survey instrument, librarians and students reported their experience, usage, and opinions of the system. Upon responding to process-related questions about a search simulation included in the questionnaire, participants described and illustrated their understanding of MCS operation. Data show that students considered MCS primarily as a tool for locating full text, while librarians viewed it as a secondary search tool with disappointing performance. In discussing MCS operation, students focused largely on full-text retrieval capability and search efficiency whereas librarians paid more attention to search strategies and retrieval quality. Both students and librarians indicated that the lack of background information about MCS operation was problematic. Both groups identified knowledge and skills related to federated searching for information literacy programs. Implications of the findings for information literacy, interface design, and system development were explored.     

Usability Testing: A Journey, Not a Destination

Usability testing for interfaces can fit into a design project in several ways. The author looks at how good testing techniques can make projects run better and faster, and produce an improved result. The primary point is that there's no incompatibility between usability testing and a rational product process. We can think of usability testing as simply another aspect of quality assurance, and can fold it into the product process in much the same way as software quality assurance (QA). By having usability people involved from the beginning, big problems get found earlier, and small problems get found in time. The fear of losing control of a project to a usability person who finds a last-minute interface design problem is therefore a red herring. You can indeed find usability problems that might affect project completion late in the development process, just like you can find system architecture problems. But the way to keep this from happening is to test your system and your ideas during each stage of development, gathering the kind of information you need at each of those stages. That's why usability testing is a journey, not a destination     

Design and evaluation of a dynamic job search website for non-literate users of Pakistan

ABSTRACT

Among the many available online job searching solutions, very limited are proposed for non-literate users. The available solutions have tested the possibilities using static web pages which are not applicable in the real world for non-literate users because of the dynamism of the real world. In this research, a dynamic job search solution is presented for non-literate users of Pakistan. User Interface (UI) requirements were identified through an extensive literature review. The culturally specific content in the form of hand-drawn sketches, voice feedback, semi-illustrations was identified and incorporated in the job seekers (non-literate) module. The same module was evaluated by the non-literate and semi-literate participants in a field study as per the usability specification defined by ISO-9241-11. The results have shown that non-literate users could search for a job efficiently in the minimum task completion time as compared to the time reported in the previous literature. They were satisfied and motivated to use the website again to search for jobs. The results indicate a potential for designing ICT-based solutions to provide services to the non-literate users in developing countries.     

Comparative usability measurement: the role of the usability lab in PC Magazine UK and PC/ Computing

Abstract

PC Magazine UK and PC/Computing have pioneered the development of formal comparative usability testing, in response to the changing objectives and requirements of their readership. The comparative methodology is labs-based and follows a strict series of procedures during the design, implementation and analysis of tests. Carefully selected testers work with the products under review and subjective and objective measures are recorded, from which a comparison of all products can be derived. The end result of the comparative methodology is the ranking, or sorting, of products within the same category, in order of their usability. This information is presented to readers as a basis for product choice; and to vendors as a guide to the strengths and weaknesses of their product in relation to its competitors.