Some surprising differences between novice and expert errors in computerized office work

This paper investigates the impact of different levels of expertise on errors in human-computer interaction. In a field study 174 clerical workers from 12 different companies were observed during their normal office work and were questioned on their expertise with computers. The level of expertise was determined by (a) the length of time an employee had worked with a computer (computer expertise); (b) the number of programs she knew (program expertise); and (c) the daily time s/he spent working with the computer (daily work-time expertise). These different operationalizations of novices and experts led to different results. In contrast to widespread assumptions, experts did not make fewer errors than novices (except in knowledge errors). On the other hand, experts spent less time handling the errors than novices. A cluster analysis produced four groups in the workforce: occasional users, frequent users, beginners, and general users.     

Influences of general computer experience and age on library database search performance

Two experiments examined the effects of general computer experience and age on library system search performance among novice library system users. Twenty younger adults (10 with high and 10 with low computer experience) and 20 older adults (10 with low and 10 with no computer experience) performed 10 search tasks of varying difficulty. Search success, syntax errors, database field specifications, keyword specifications, and use of Boolean operators were examined. Among younger novices, high computer experience was associated with slightly better performance than low computer experience. Among older novices, having some computer experience was associated with much better performance than no computer experience. Older computer users showed lower overall success rates, made more syntax and field specification errors, and demonstrated poorer understanding of Boolean logic and keyword matching algorithms than younger adults with similar computer experience. Implications for interface design and training interventions for novice on-line library system users are discussed.     

What does it mean to be good at using a mobile device? An investigation of three levels of experience and skill

An increasing number of computer users lack formal training in operating their devices. These daily users cannot be described as novices or experts within the predominant view of expertise. In order to describe and better understand this type of self-taught intermediate level of skill, 10 casual users of a high-end smartphone series were compared to 10 novices and 4 professionals (help desk personnel) in their learning histories, task performance, and cognitive outcomes. Our study suggests that this type of self-taught intermediate level of skill is device-specific. Experienced users (casual users and experts) exhibited superior performance for representative tasks. This is mainly attributable to faster navigation and better knowledge of interface terminology, not to deeper conceptual representation of the problems. Interviews suggest that this skill is the consequence of routine use and three recurring learning events: familiarization, following of media, and ad hoc problem-solving situations. We conclude by discussing why intermediate levels of skill deserve more attention in HCI research.     

Attention,polychronicity, and expertise in prospective memory performance: Programmers' vulnerability to habit intrusion error in multitasking

The paper examines prospective memory (ProM) in programmer multitasking and reports administration of a naturalistic atypical action. The study emphasizes on how attention, time orientation, and expertise affect ProM performance in multitasking among a group of computer science and engineering students (N=108). The results suggest that attention play a crucial role in multitasking and ProM performance with respect to whether or not a word display requires more attention to be devoted in monitoring and identifying it for an appropriate action. Polychrons exhibit lesser degree of ProM performance failure than monochrons whereas expertise does not have an effect. Finally, results show that out of overall ProM performance failure, habit intrusion errors comprise of 16.22% occurring 1.75 times in every 10 valid click responses of ProM task. Moreover, experts demonstrate a superior performance over novices in programming.     

The influence of user expertise and phone complexity on performance,ease of use and learnability of different mobile phones

The study focuses on usability, ease of use and learnability of three different mobile phones (Nokia 3210, Siemens C35i, Motorola P7389). The first independent variable refers to the complexity of the menu (depth/breadth of the menu tree) and navigation keys (number/functionality). The Nokia phone had the lowest and the Motorola the highest complexity, with the Siemens phone ranging between them. The second independent variable was user expertise: 30 novices and 30 experts solved six telephone tasks. In order to assess effects of learnability, tasks were presented twice. Differences between the mobile phones regarding effectiveness, efficiency and learnability were found: The best performance was shown by Nokia users. The remaining two phones did not differ significantly, although the most complex phone was superior to the phone of medium complexity which had the lowest performance. Moreover, an effect of expertise was confirmed, though suboptimal interfaces were identified as lessening the advantage of expertise. Specific weaknesses of the tested phones are discussed.     

Expert versus novice use of the executive support systems: an empirical study

Expertise is essential in problem solving, particularly for executives. Much literature indicates that, in solving daily problems, executives may employ both analytical thinking and intuition. To develop skills and apply intuition, company executives require the use of support devices such as executive support systems (ESS), but their value may differ substantially between experts and novices.Previous research has failed to address the effects of expertise on ESS use. Therefore, this study examines how professionals use them to address business-oriented tasks. The professionals studied were divided into two groups, depending on whether they were experts or novices in the business represented by those tasks. The findings of the experiments show that computer self-efficacy strongly influenced the use of ESS. Moreover, experts felt that the ESS was of greater use when more powerful systems were employed, while the difference for novices between more and less powerful systems was insignificant. Finally, experts rated their user information satisfaction score significantly higher when employing more powerful systems. However, novices rated systems significantly higher only when they cognitively fit the performed task. Implications are discussed.     

Influences of general computer experience and age on library database search performance

Two experiments examined the effects of general computer experience and age on library system search performance among novice library system users. Twenty younger adults (10 with high and 10 with low computer experience) and 20 older adults (10 with low and 10 with no computer experience) performed 10 search tasks of varying difficulty. Search success, syntax errors, database field specifications, keyword specifications, and use of Boolean operators were examined. Among younger novices, high computer experience was associated with slightly better performance than low computer experience. Among older novices, having some computer experience was associated with much better performance than no computer experience. Older computer users showed lower overall success rates, made more syntax and field specification errors, and demonstrated poorer understanding of Boolean logic and keyword matching algorithms than younger adults with similar computer experience. Implications for interface design and training interventions for novice on-line library system users are discussed.     

The influence of user expertise and prototype fidelity in usability tests

An empirical study examined the impact of user expertise and prototype fidelity on the outcomes of a usability test. User expertise (expert vs. novice) and prototype fidelity (paper prototype, 3D mock-up, and fully operational appliance) were manipulated as independent variables in a 2 × 3 between-subjects design. Employing a floor scrubber as a model product, 48 users carried out several cleaning tasks. Usability problems identified by participants were recorded. Furthermore, performance, system management strategies and perceived usability were measured. The results showed that experts reported more usability problems than novices but these were considered to be less severe than those reported by novices. Reduced fidelity prototypes were generally suitable to predict product usability of the real appliance. The implications for the running of usability tests are specific to the fidelity of the prototype.     

The Nature of Expertise in Scheduling: The Case of Timetabling

This study aims to gain greater insight into scheduling expertise by comparing the work of experts and novices when designing a university timetable. We assumed that the scheduling activity would take place within two dual spaces: the constraints space (CS) and the objects space (OS). Constraints are defined in the strictest sense as relations between variables that cannot be represented in the solution (timetable), whereas objects are constraint satisfactions that can be thus represented. The study shows that experts were more likely than novices to use external representations as activity support. They satisfied many constraints with partially defined objects. On the contrary, novices devoted a long time to managing constraints in their heads before defining only fully specified objects (concrete objects). The objects space could be a suitable activity support for experts. Novices, on the other hand, could benefit from support in managing constraints and translating constraints into objects. © 2012 Wiley Periodicals, Inc.     

Biomechanical differences between expert and novice workers in a manual material handling task

The objective was to verify whether the methods were safer and more efficient when used by expert handlers than by novice handlers. Altogether, 15 expert and 15 novice handlers were recruited. Their task was to transfer four boxes from a conveyor to a hand trolley. Different characteristics of the load and lifting heights were modified to achieve a larger variety of methods by the participants. The results show that the net moments at the L5/S1 joint were not significantly different (p > 0.05) for the two groups. However, compared with the novices, the experts bent their lumbar region less (experts 54° (SD 11°); novices 66° (SD 15°)) but bent their knees more (experts approx. 72° (SD approx. 30°); novices approx. 53° (SD approx. 33°), which brought them closer to the box. The handler's posture therefore seems to be a major aspect that should be paid specific attention, mainly when there is maximum back loading.

Statement of Relevance: The findings of this research will be useful for improving manual material handling training programmes. Most biomechanical research is based on novice workers and adding information about the approach used by expert handlers in performing their tasks will help provide new avenues for reducing the risk of injury caused by this demanding physical task.     

Intensive use of ICT in school: Developing differences in students’ ICT expertise

The purpose of the study was to examine the development of students’ high-level computer skills and competence (student expertise) in information and communication technology (ICT), and to examine the characteristics of such expertise. Eighteen lower secondary school students, selected to represent both genders and all school achievement levels, were given laptops for three years to be used both at school and at home. The data of the longitudinal study consisted of a test on ICT skills; self-evaluation questionnaires on ICT competence, thoughts about their own expertise, ICT-related activities and task, interest in ICT, and classroom observations. The results of the study indicated that the intensive use of ICT and the process-oriented learning environment supported the development of student expertise. In the analysis, three groups were identified, which oriented themselves somewhat differently in relation to ICT: student experts (n = 6), advanced users (n = 8) and non-interested users (n = 4). The experts differed from the other groups in respect of especially focusing on advancing their ICT skills; they had undertaken ICT-related tasks outside the school, they had metacognitive consciousness about their competence, and they had future plans concerning ICT in their further education and profession.     

Quantitative and qualitative differences between experts and novices in chunking computer software knowledge

This study investigated quantitative and qualitative differences between experts and novices in knowledge structure, specifically in their chunking of computer software knowledge. An experiment was conducted to collect pairwise relatedness ratings of 23 concepts in C computer programming from 10 experts and 10 novices. Correlation analyses of relatedness ratings were performed to examine expert‐novice differences in overall knowledge structure. The intragroup correlation coefficients of experts were greater than the intragroup correlation coefficients of novices and the intergroup correlation coefficients between experts and novices. Hence, there existed structural differences of knowledge between experts and novices. To examine how experts differ from novices quantitatively and qualitatively in their knowledge chunking, the relatedness ratings of experts were averaged within the group to give a single set of relatedness ratings, and the relatedness ratings of novices were also averaged within the group to give a single set of relatedness ratings. Then the hierarchical clustering analyses were performed on the set of relatedness ratings for the experts group and on the set of relatedness ratings for the novices group, respectively. The hierarchical clustering of C concepts made by experts and the hierarchical clustering of C concepts made by novices indicated that novices had more but smaller knowledge chunks than experts for the common set of knowledge, and thatthe rough chunking of knowledge made by novices was qualitatively similar to the rough chunking of knowledge made by experts, although on a more detailed level qualitative differences still existed.     

Interacting with notebook input devices: an analysis of motor performance and users' expertise

In the present study the usability of two different types of notebook input devices was examined. The independent variables were input device (touchpad vs. mini-joystick) and user expertise (expert vs. novice state). There were 30 participants, of whom 15 were touchpad experts and the other 15 were mini-joystick experts. The experimental tasks were a point-click task (Experiment 1) and a point-drag-drop task (Experiment 2). Dependent variables were the time and accuracy of cursor control. To assess carryover effects, we had the participants complete both experiments, using not only the input device for which they were experts but also the device for which they were novices. Results showed the touchpad performance to be clearly superior to mini-joystick performance. Overall, experts showed better performance than did novices. The significant interaction of input device and expertise showed that the use of an unknown device is difficult, but only for touchpad experts, who were remarkably slower and less accurate when using a mini-joystick. Actual and potential applications of this research include an evaluation of current notebook input devices. The outcomes allow ergonomic guidelines to be derived for optimized usage and design of the mini-joystick and touchpad devices.     

The Expertise Effect on Web Accessibility Evaluation Methods

Web accessibility means that disabled people can effectively perceive, understand, navigate, and interact with the web. Web accessibility evaluation methods are needed to validate the accessibility of web pages. However, the role of subjectivity and of expertise in such methods is unknown and has not previously been studied. This article investigates the effect of expertise in web accessibility evaluation methods by conducting a Barrier Walkthrough (BW) study with 19 expert and 57 nonexpert judges. The BW method is an evaluation method that can be used to manually assess the accessibility of web pages for different user groups such as motor impaired, low vision, blind, and mobile users.

Our results show that expertise matters, and even though the effect of expertise varies depending on the metric used to measure quality, the level of expertise is an important factor in the quality of accessibility evaluation of web pages. In brief, when pages are evaluated with nonexperts, we observe a drop in validity and reliability. We also observe a negative monotonic relationship between number of judges and reproducibility: more evaluators mean more diverse outputs. After five experts, reproducibility stabilizes, but this is not the case with nonexperts. The ability to detect all the problems increases with the number of judges: With 3 experts all problems can be found, but for such a level 14 nonexperts are needed. Even though our data show that experts rated pages differently, the difference is quite small. Finally, compared to nonexperts, experts spent much less time and the variability among them is smaller, they were significantly more confident, and they rated themselves as being more productive. The article discusses practical implications regarding how BW results should be interpreted, how to recruit evaluators, and what happens when more than one evaluator is hired.

Supplemental materials are available for this article. Go to the publisher's online edition of Human–Computer Interaction for statistical details and additional measures for this article.     

Expert-novice knowledge of computer programming at different levels of abstraction

Computer programming knowledge can be classified into five levels of abstraction: objective, conceptual, functional, logical, and physical. An experiment was carried out to determine whether the mastering of knowledge at different levels of abstraction changed with the level of skill. Ten experts and ten novices in C computer programming participated in the experiment. The subjects' knowledge at the five levels of abstraction was tested through 20 multiple-choice questions. The experimental results indicated that knowledge differences between experts and novices at an abstract level or a concrete level depended on what abstract or concrete knowledge was implied. Experts had better abstract knowledge than novices at the conceptual and functional levels but not at the objective level. Experts had better concrete knowledge than novices at the physical level but not at the logical level. The classification of computer programming knowledge in levels of abstraction and the experimental results helped in clarifying a general finding from previous studies that experts had better abstract knowledge than novices.     

SPEAR: SPAMMING‐RESISTANT EXPERTISE ANALYSIS AND RANKING IN COLLABORATIVE TAGGING SYSTEMS

In this article, we discuss the notions of experts and expertise in resource discovery in the context of collaborative tagging systems. We propose that the level of expertise of a user with respect to a particular topic is mainly determined by two factors. First, an expert should possess a high‐quality collection of resources, while the quality of a Web resource in turn depends on the expertise of the users who have assigned tags to it, forming a mutual reinforcement relationship. Second, an expert should be one who tends to identify interesting or useful resources before other users discover them, thus bringing these resources to the attention of the community of users. We propose a graph‐based algorithm, SPEAR (spamming‐resistant expertise analysis and ranking), which implements the above ideas for ranking users in a folksonomy. Our experiments show that our assumptions on expertise in resource discovery, and SPEAR as an implementation of these ideas, allow us to promote experts and demote spammers at the same time, with performance significantly better than the original hypertext‐induced topic search algorithm and simple statistical measures currently used in most collaborative tagging systems.     

Pull-down versus traditional menu types: An empirical comparison

An experiment was conducted to test the performance of pull-down versus traditional or explicit menus. Sixty subjects, including novice and experienced computer users, manipulated both types of menus to complete banking tasks similar to those found on Automatic Teller M achines. The order of the menus was randomly varied to control for learning effects. Across both types of users, traditional-style menus elicited fewer errors than did pull-down menus; however, no significant difference was found in the time to complete the banking task. Experienced users outperformed novice users in the amount of time taken to complete the task regardless of menu type, though no difference was found in the number of errors committed by both user types.     

Neither novice nor expert: the discretionary user of software

Most studies that examine users' interaction with computers focus either on novices or experts. Discretionary users of computers, such as lawyers, executives, administrators and professors have been ignored. Using techniques of process analysis, this study investigated the characteristics of discretionary users by observing their interaction with commercial word-processing software. Their performance was compared to groups of novices and experts. Results indicate that discretionary users exhibit expert-like characteristics on a small set of routine editing tasks, beyond which their behavior is quite novice-like. A procedural model that indicates areas where software design could improve the editing performance of discretionary users is also discussed.     

Environments to support context and emotion aware visual interaction

The interaction with software systems is often affected by many types of hurdles that induce users to make errors and mistakes, and to break the continuity of their reasoning while carrying out a working task with the computer. As a consequence, negative emotional states, such as frustration, dissatisfaction, and anxiety, may arise. In this paper, we illustrate how the Software Shaping Workshop (SSW) methodology can represent a solution to the problem of developing interactive systems that are correctly perceived and interpreted by end-users, thus becoming more acceptable and favouring positive emotional states. In the methodology, a key role is played by domain-expert users, that is, experts in a specific domain, not necessarily experts in computer science. Domain-expert users’ skills and background, including their knowledge of the domain and users’ needs and habits, are exploited to create context and emotion aware visual interactive systems. Examples of these systems are illustrated by referring to a case study in the automation field.     

The long-term effects of integral versus composite metaphors on experts' and novices' search behaviors

For years, metaphors have been used extensively to facilitate multiple user tasks on Web sites. Nonetheless, research examining metaphor's effects in facilitating user tasks, not to mention the proposal of the design methodology, is limited. There is disagreement about using single or multiple metaphors in designing computer systems. Regarding metaphor's long-term effects, Carroll and Thomas [Carroll, J.M., Thomas, C.J., 1982. Metaphor and the cognitive representation of computing systems. IEEE Transactions on Systems, Man, and Cybernetics, SMC 12, 107–116] claim that metaphors lose utility once users are familiar with the system. Furthermore, little evidence exists about metaphor's effects on users with differing computer experience.This researcher proposes a metaphor design methodology to examine the long-term effects of integral (single) versus composite (mixed) metaphors on subjects' information search behaviors and their effects on Internet novices and experts. Ninety-eight college students participated in this study. The findings suggest that metaphor's effects decrease over time as users become more experienced. Moreover, composite metaphors facilitate experts' searching better than novices'.