How natural is a natural interface? An evaluation procedure based on action breakdowns

This paper describes an issue-based method to evaluate the naturalness of an interface. The method consists of the execution of a series of tasks on that interface, which is subsequently systematically analyzed to identify breakdowns in the users’ actions. The systematic analysis of breakdowns is allowed by the support of video-coding software (The Observer by Noldus). This method is described on its theoretical bases and then applied to the evaluation of a natural interface, a walk-in-place locomotion system for virtual spaces called Superfeet. The procedure is comparative, since Superfeet is compared to two locomotion devices, Superfeet enhanced with headtracker and a more traditional Joypad. The test involves 36 participants (mean age = 23.68, SD = 3.14). The outcomes of the breakdown analysis are illustrated at a progressively finer level of granularity from the amount and length of breakdowns, to the circumstances of the breakdowns, to the type of actions involved in the breakdowns. The potential of this procedure for usability studies is finally synthesized.     

Dual Verbal Elicitation: The Complementary Use of Concurrent and Retrospective Reporting Within a Usability Test

Verbal protocols are the primary tool for understanding users' task-solving behaviors during usability testing. A qualitative study that examined the utility of combining a concurrent and retrospective think-aloud within the same usability test is described. The results indicate that although there was significant overlap between the types of utterances produced during each think-aloud, the retrospective phase produced more verbalizations that were relevant to usability analysis, for example, helpful self-assessments of performance, yielding insights into the impact of encountered difficulties. However, a small number of less desirable utterance types emerged: hypothesising, rationalizing, and forgetting. When used together, both methods contributed to an understanding of usability issues; the concurrent phase yielded more usability issues overall, and the retrospective data improved the understanding of these by (a) reinforcement: users highlighted the impact of an issue on their experience, (b) elaboration: users would provide causal explanations of encountered difficulties, and (c) context: users provided information about the product's context of use.     

Construction of interaction observation systems for collaboration analysis in groupware applications

Interaction observation systems for groupware applications capture and process all the actions performed by users engaged in workgroups. These actions are then stored in log documents that enable the work process carried out by the users to be analyzed and the interaction between users to be studied. This article proposes an approach, based on ontological models, which is devised to help the developer of an observation system for a groupware application to structure and record user actions. In order to achieve this aim, we present a specific ontology that shapes the collaborative work process of the users so as to obtain an XML-based log document that stores all the actions carried out by the users and facilitates the subsequent analysis of the system usage and users’ behavior. This approach has been used to improve communication and collaboration capabilities in the COLLECE groupware application.     

Workflow Systems: Occasions for Success and Failure

Workflow technologies have created considerable discussion within the computer supported cooperative work community. Although a number of theoretical and empirical warnings about the difficulties of workflow systems have appeared, the technologies continue to be built and sold. This paper examines the use of one workflow-like system and outlines three cases when the technology supported the work of its users. Comparing these successful occasions with some reports of difficulties, this paper draws conclusions about the circumstances that led to tool usage.     

Beyond language translation: crossing the cultural divide

Working in and designing for other cultures con lead to communication breakdowns. The author explores how this happens and suggests ways that we con use potentially problematic situations to our advantage. Although it is fundamentally difficult to establish successful cross-cultural collaboration, we should not avoid it. On the contrary, we should exploit cross-cultural work opportunities: encountering cultural breakdowns is the best way to widen our perspectives, understand ourselves more deeply, and consequently, be more creative, Finally. if communication breakdowns are unavoidable, why not use them? These breakdowns can be a driving force for coordinating cross-cultural collaboration-whether among people from different nations or different work cultures     

Virtual team collaboration: building shared meaning, resolving breakdowns and creating translucence

Managing international teams with geographically distributed participants is a complex task. The risk of communication breakdowns increases due to cultural and organizational differences grounded in the geographical distribution of the participants. Such breakdowns indicate general misunderstandings and a lack of shared meaning between participants. In this paper, we address the complexity of building shared meaning. We examine the communication breakdowns that occurred in two globally distributed virtual teams by providing an analytical distinction of the organizational context as the foundation for building shared meaning at three levels. Also we investigate communication breakdowns that can be attributed to differences in lifeworld structures, organizational structures, and work process structures within a virtual team. We find that all communication breakdowns are manifested and experienced by the participants at the work process level; however, resolving breakdowns may require critical reflection at other levels. Where previous research argues that face-to-face interaction is an important variable for virtual team performance, our empirical observations reveal that communication breakdowns related to a lack of shared meaning at the lifeworld level often becomes more salient when the participants are co-located than when geographically distributed. Last, we argue that creating translucence in communication structures is essential for building shared meanings at all three levels.     

Analyzing government regulations using structural and domain information

Government regulations, by extending laws with specific guidance for corporate and public actions, provide an important societal benefit. Ideally, they should be intelligible to ordinary citizens as well as rule makers, but the volume of regulations coupled with heavy referencing between provisions limit their accessibility. Apart from the difficulties in locating and understanding a particular regulation, users often must consult and reconcile multiple authoritative sources. For example, US companies frequently must comply with overlapping federal, state, and local regulations; in addition, some nonprofit organizations publish their own codes of practice. The problem is exacerbated in the European Union, where regulators must harmonize legislation across countries with different languages and traditions. To address the difficulties encountered in comparing regulatory documents with multiple authoritative sources, the Regnet project is developing a relatedness analysis system that exploits such document's unique computational properties.     

Team play with a powerful and independent agent: operational experiences and automation surprises on the Airbus A-320

Research and operational experience have shown that one of the major problems with pilot-automation interaction is a lack of mode awareness (i.e., the current and future status and behavior of the automation). As a result, pilots sometimes experience so-called automation surprises when the automation takes an unexpected action or fails to behave as anticipated. A lack of mode awareness and automation surprises can he viewed as symptoms of a mismatch between human and machine properties and capabilities. Changes in automation design can therefore he expected to affect the likelihood and nature of problems encountered by pilots. Previous studies have focused exclusively on early generation "glass cockpit" aircraft that were designed based on a similar automation philosophy. To find out whether similar difficulties with maintaining mode awareness are encountered on more advanced aircraft, a corpus of automation surprises was gathered from pilots of the Airbus A-320, an aircraft characterized by high levels of autonomy, authority, and complexity. To understand the underlying reasons for reported breakdowns in human-automation coordination, we also asked pilots about their monitoring strategies and their experiences with and attitude toward the unique design of flight controls on this aircraft.     

Privacy in instant messaging: an impression management model

Instant messaging (IM) has evolved into an important tool for collaborative work. It supports informal near-synchronous communication and fosters awareness of the online presence of one's communication partners. Like all awareness systems, IM runs into concerns regarding privacy. Drawing upon prior literature and exploratory interviews, we postulate a model that posits impression management as an underlying cause for privacy desires of IM users. We verify our hypotheses using linear structural modelling on data from a large online survey of IM users across the US. The model establishes that the desire for privacy in IM arises due to the desire for impression management (both directly, as well as indirectly through the desire for visibility of one's impression to oneself). Based on this model, we suggest that IM systems could support privacy needs of users better by providing them with more knowledge and control over aspects that affect their IM-conveyed impression on others (i.e. by making impression management functionality available). Specifically, to help convey and sustain appropriate impressions on IM contacts, IM systems should allow for increased visibility of one's actions to oneself, facilitate easy comparison of one's practices with those of others, and allow one to view oneself from the perspective of others and to make finer-grained adjustments to IM settings than is possible today.     

Tool-provisioning strategies for flexible manufacturing systems

The difficulties encountered in managing the tool flow in flexible manufacturing systems for the manufacture of prismatic parts indicate the requirement to handle operational issues such as tool assignment. The choice of operational strategy and its relationship to machine specification, work and tool loading, scheduling and specific mode of tool management may significantly enhance machine utilization and work throughput. In this paper, strategies are presented aimed at improving system efficiency and minimizing tooling costs by considering tool provision as work-orientated, where the tooling is changed to suit the production task, or tool-orientated, where resident tools dictate the work flow, or a combination of the two. This paper draws on evidence from current industrial practice and recent developments.     

Providing adaptation and guidance for design learning by problem solving: The design planning approach in DomoSim-TPC environment

Experimental learning environments based on simulation usually require monitoring and adaptation to the actions the users carry out. Some systems provide this functionality, but they do so in a way which is static or cannot be applied to problem solving tasks. In response to this problem, we propose a method based on the use of intermediate languages to provide adaptation in design learning scenarios. Although we use some approaches which are familiar from other domains (e.g., programming tutors) they are novel as regards their application to a very different domain and as a result we have incorporated new strategies. The purpose of our proposal is to provide monitoring, guidance and adaptive features for PlanEdit, a tool for the learning of integral automation methods in buildings and housing by design. This tool is part of a collaborative environment, called DomoSim-TPC, which supports distance learning of domotical design. We have carried out an experiment to obtain some data which confirm that our position can be effective for group learning of domotical design, studying the relationship between the quantity of model work carried out and the errors made.     

Web user profiles with time-decay and prototyping

User profiling represents an important initial step in personalizing web services and in building recommendation systems. Non-invasive profiling methods monitor users’ behavior and infer interest profiles from their past actions. Most existing profiling methods, which relate the users’ interests to a given ontology, consider only the user’s past actions when calculating his/her profile. The profiling algorithms use a time-decay function for users’ past actions to adapt the profile to shifts in the user’s interests over time. In our work, we propose a hybrid method that combines time-decay and profile correction using prototype profiles. The additional profile correction step considers the interests of similar users and expands the interest scores beyond the concepts detected in the user’s past actions, which facilitates faster profile adaptation to the user’s new interests. In our experimental work, we experimented extensively with two real data sets: data of an online advertising network and student data in an online e-learning environment. We measured the quality of the computed user profiles by correlating them to users’ future actions. Experiments revealed that it is crucial to build the user’s profile using a large number of events from his/her past and to update the profile regularly. When we are unable to do so, the profile correction can be used to keep the quality of the profile from dropping too low. The results show that our method significantly outperforms existing ontological profiling methods.     

Ethnography, customers, and negotiated interactions at the airport

In the late 1990s, tightly coordinated airline schedules unraveled owing to massive delays resulting from inclement weather, overbooked flights, and airline operational difficulties. As schedules slipped, the delayed departures and late arrivals led to systemwide breakdowns, customers missed their connections, and airline work activities fell further out of sync. In offering possible answers, we emphasize the need to consider the customer as participant, following the human-centered computing model. Our study applied ethnographic methods to understand the airline system domain and the nature of airline delays, and it revealed the deficiencies of the airline production system model of operations. The research insights that led us to shift from a production and marketing system perspective to a customer-as-participant view might appear obvious to some readers. However, we do not know of any airline that designs its operations and technologies around any other model than the production and marketing system view. Our human-centered analysis used ethnographic methods to gather information, offering new insight into airline delays and suggesting effective ways to improve operations reliability.     

A model of keyboard configuration requirements

This paper presents a user model: a computer program which examines the behaviour of a real computer user. The model encompasses four aspects of keyboard use which can present difficulties for people with motor disabilities. Where relevant keyboard configuration options exist, the model chooses appropriate settings for these options. The model bases its recommendations on observation of users typing free English text. It is intended to form part of a dynamic configuration support tool. Empirical evaluation showed the model to be very accurate in identification of a given user's difficulties. Where recommended configuration options were tried by the participants, high levels of error reduction and user satisfaction were found.     

Implementing and evaluating scheduling policies in gLite middleware

Grid scheduling algorithms are usually implemented in a simulation environment using tools that hide the complexity of the Grid and assumptions that are not always realistic. In our work, we describe the steps followed, the difficulties encountered and the solutions provided to develop and evaluate a scheduling policy, initially implemented in a simulation environment, in the gLite Grid middleware. Our focus is on a scheduling algorithm that allocates in a fair way the available resources among the requested users or jobs. During the actual implementation of this algorithm in gLite, we observed that the validity of the information used by the scheduler for its decisions affects greatly its performance. To improve the accuracy of this information, we developed an internal feedback mechanism that operates along with the scheduling algorithm. Also, a Grid computation resource cannot be shared concurrently between different users or jobs, making it difficult to provide actual fairness. For this reason we investigated the use of virtualization technology in the gLite middleware. We did a proof‐of‐concept implementation and performed an experimental evaluation of our scheduling algorithm in a small gLite testbed that proves the validity and applicability of our solutions. Copyright © 2012 John Wiley & Sons, Ltd.     

Identifying problems and generating recommendations for enhancing complex systems: applying the abstraction hierarchy framework as an analytical tool

OBJECTIVE: This study adopts J. Rasmussen's (1985) abstraction hierarchy (AH) framework as an analytical tool to identify problems and pinpoint opportunities to enhance complex systems. BACKGROUND: The process of identifying problems and generating recommendations for complex systems using conventional methods is usually conducted based on incompletely defined work requirements. As the complexity of systems rises, the sheer mass of data generated from these methods becomes unwieldy to manage in a coherent, systematic form for analysis. There is little known work on adopting a broader perspective to fill these gaps. METHOD: AH was used to analyze an aircraft-automation system in order to further identify breakdowns in pilot-automation interactions. Four steps follow: developing an AH model for the system, mapping the data generated by various methods onto the AH, identifying problems based on the mapped data, and presenting recommendations. RESULTS: The breakdowns lay primarily with automation operations that were more goal directed. Identified root causes include incomplete knowledge content and ineffective knowledge structure in pilots' mental models, lack of effective higher-order functional domain information displayed in the interface, and lack of sufficient automation procedures for pilots to effectively cope with unfamiliar situations. CONCLUSION: The AH is a valuable analytical tool to systematically identify problems and suggest opportunities for enhancing complex systems. It helps further examine the automation awareness problems and identify improvement areas from a work domain perspective. APPLICATION: Applications include the identification of problems and generation of recommendations for complex systems as well as specific recommendations regarding pilot training, flight deck interfaces, and automation procedures.     

What we know about dyslexia and Web accessibility: a research review

Compared to the online interaction behavior of other users, little is known about the difficulties dyslexic Web users encounter online. This paper reviews existing literature at the intersection of dyslexia and accessibility research to determine what useful knowledge exists regarding this important and relatively large group of users. This review uncovers that, although there are few published usability tests with dyslexic users, there is a considerable body of knowledge on dyslexia as well as many design guidelines for authoring dyslexic-accessible interfaces. Through a comparison of existing accessibility guidelines for dyslexic and non-dyslexic users and discussion of the plain language movement, it is argued that dyslexic-accessible practices may redress difficulties encountered by all Internet users. This conclusion suggests that usability testing yielding a clearer profile of the dyslexic user would further inform the practice of universal design, but also that enough knowledge is already available to allow doing more to increase accessibility for dyslexic Internet users.     

Prediction of individual learning curves across information visualizations

Confident usage of information visualizations is thought to be influenced by cognitive aspects as well as amount of exposure and training. To support the development of individual competency in visualization processing, it is important to ascertain if we can track users’ progress or difficulties they might have while working with a given visualization. In this paper, we extend previous work on predicting in real time a user’s learning curve—a mathematical model that can represent a user’s skill acquisition ability—when working with a visualization. First, we investigate whether results we previously obtained in predicting users’ learning curves during visualization processing generalize to a different visualization. Second, we study to what extent we can make predictions on a user’s learning curve without information on the visualization being used. Our models leverage various data sources, including a user’s gaze behavior, pupil dilation, and cognitive abilities. We show that these models outperform a baseline that leverages knowledge on user task performance so far. Our best performing model achieves good accuracies in predicting users’ learning curves even after observing users’ performance on a few tasks only. These results represent an important step toward understanding how to support users in learning a new visualization.