Hands-free administration of subjective workload scales: Acceptability in a surgical training environment

Introduction

Subjective workload measures are usually administered in a visual-manual format, either electronically or by paper and pencil. However, vocal responses to spoken queries may sometimes be preferable, for example when experimental manipulations require continuous manual responding or when participants have certain sensory/motor impairments. In the present study, we evaluated the acceptability of the hands-free administration of two subjective workload questionnaires - the NASA Task Load Index (NASA-TLX) and the Multiple Resources Questionnaire (MRQ) - in a surgical training environment where manual responding is often constrained.

Method

Sixty-four undergraduates performed fifteen 90-s trials of laparoscopic training tasks (five replications of 3 tasks - cannulation, ring transfer, and rope manipulation). Half of the participants provided workload ratings using a traditional paper-and-pencil version of the NASA-TLX and MRQ; the remainder used a vocal (hands-free) version of the questionnaires. A follow-up experiment extended the evaluation of the hands-free version to actual medical students in a Minimally Invasive Surgery (MIS) training facility.

Results

The NASA-TLX was scored in 2 ways - (1) the traditional procedure using participant-specific weights to combine its 6 subscales, and (2) a simplified procedure - the NASA Raw Task Load Index (NASA-RTLX) - using the unweighted mean of the subscale scores. Comparison of the scores obtained from the hands-free and written administration conditions yielded coefficients of equivalence of r = 0.85 (NASA-TLX) and r = 0.81 (NASA-RTLX). Equivalence estimates for the individual subscales ranged from r = 0.78 (“mental demand”) to r = 0.31 (“effort”). Both administration formats and scoring methods were equally sensitive to task and repetition effects. For the MRQ, the coefficient of equivalence for the hands-free and written versions was r = 0.96 when tested on undergraduates. However, the sensitivity of the hands-free MRQ to task demands (η_partial² = 0.138) was substantially less than that for the written version (η_partial² = 0.252). This potential shortcoming of the hands-free MRQ did not seem to generalize to medical students who showed robust task effects when using the hands-free MRQ (η_partial² = 0.396). A detailed analysis of the MRQ subscales also revealed differences that may be attributable to a “spillover” effect in which participants’ judgments about the demands of completing the questionnaires contaminated their judgments about the primary surgical training tasks.

Conclusion

Vocal versions of the NASA-TLX are acceptable alternatives to standard written formats when researchers wish to obtain global workload estimates. However, care should be used when interpreting the individual subscales if the object is to make comparisons between studies or conditions that use different administration modalities. For the MRQ, the vocal version was less sensitive to experimental manipulations than its written counterpart; however, when medical students rather than undergraduates used the vocal version, the instrument’s sensitivity increased well beyond that obtained with any other combination of administration modality and instrument in this study. Thus, the vocal version of the MRQ may be an acceptable workload assessment technique for selected populations, and it may even be a suitable substitute for the NASA-TLX.     

The Adult Attachment Interview and self-reports of attachment style: An empirical rapprochement.

Although 10 studies have been published on the empirical overlap of the Adult Attachment Interview (AAI) and measures of self-reported attachment style, results in this literature have been inconsistently interpreted in narrative reviews. This report was designed as a rapprochement of the AAI and attachment style literatures and includes 3 studies. Study 1 (combined N = 961) is a meta-analytic review showing that by J. Cohen's (1992) criteria (mean r = .09), the association between AAI security and attachment style dimensions is trivial to small. Study 2 (N = 160) confirms meta-analytic results with state-of-the-art assessments of attachment security and also examines attachment dimensions in relation to the Big 5 personality traits. Finally, Study 3 is an investigation of 50 engaged couples that shows that developmental and social psychological measures of attachment security predict somewhat distinct--though theoretically anticipated--aspects of functioning in adult relationships. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Satellite observations of circulation features associated with a bowhead whale feeding ‘hotspot’ near Barrow, Alaska

Satellite images, along with oceanographic, meteorological, and whale aerial survey data, are used to illustrate aspects of ocean circulation associated with a bowhead whale feeding ‘hotspot’ near Barrow, Alaska. In response to weak winds, a strong front forms near the shelf-break along the southern edge of Barrow Canyon when the Alaska Coastal Current flows adjacent to the southern flank of Barrow Canyon or intrudes onto the western Beaufort shelf. This front is of particular local interest because it is indicative of aggregation and retention of zooplankton on the western Beaufort shelf and, as a result, is a locus for bowhead whales pausing to feed during their westward fall migration. Groups (4 or more individuals) of bowhead whales are primarily seen on the western Beaufort shelf following wind conditions that promote the formation of this front.     

New and used temporal models: An issue of time

TheSpecial Issue on Applications of Temporal Models raises many issues of time: What are the important properties of time? How can time be best represented? How can one reason about time-dependent properties? What are the important directions of temporal research? This introductory piece very briefly surveys the current wide variety of temporal models, temporal reasoning methods, and applications to time-varying phenomena. Promising areas of investigation such as the verification of concurrent systems, knowledge-base representation methods, and dealing with theFrame Problem pass in fleeting review. Brief introductions to each of the works in the volume close the section.     

Optical constants of absorbing films

A new method for the determination of the optical constants of absorbing films supported on a transparent substrate using measured normal incidence R/T and R/T values is presented. Compared to the classical normal incidence photometric methods of deducing the optical constants from the measured R and T values the present method offers a time saving of 50% in experimental measurements and is able to give the optical constants and thickness of the films simultaneously to an accuracy of 2% and 0.5% respectively.     

Quality of Service Networking for High Performance Grid Applications

This paper reports on different efforts to provide quality of service (QoS) Networking to Grid applications done in the context of the MB-NG, GRS and DataTAG EU projects. These are leading edge network research projects involving more that 50 researchers in the UK, Europe and North America, concerned with the development and testing of protocols and standards for the next generation of high speed networks. We have implemented and tested the Differentiated Services Architecture (DiffServ) in a multi-domain, 2.5 Gbits/s network (the first such deployment) defining appropriate Service Level Agreements (SLAs) to be used between administrative domains to guarantee end-to-end Quality of Service. We characterised several hardware implementations of DiffServ and concluded on their appropriateness for several network scenarios. Since current and future Grid applications will have to use modified mechanisms of congestion control we have evaluated old and new TCP implementations over a Differentiated Services Networks. These quality of service tests have also included innovative MPLS (Multi-Protocol Label Switching) experiments to establish guaranteed bandwidth connections to Grid applications in a fast and efficient way. We have also developed a software based bandwidth broker architecture for Grids based on IETF standards which allows applications to transparently request dynamic and advanced reservations and implemented it in a real experimental network. We finally report on experiences delivering Quality of Service networking to high performance applications like Particle Physics data transfer and High Performance Computation. This includes quantitative results on the performance improvements that QoS brought to real data transfers in the context of High Performance Computing.     

Verification of evolving software via component substitutability analysis

This paper presents an automated and compositional procedure to solve the substitutability problem in the context of evolving software systems. Our solution contributes two techniques for checking correctness of software upgrades: (1) a technique based on simultaneous use of over-and under-approximations obtained via existential and universal abstractions; (2) a dynamic assume-guarantee reasoning algorithm—previously generated component assumptions are reused and altered on-the-fly to prove or disprove the global safety properties on the updated system. When upgrades are found to be non-substitutable, our solution generates constructive feedback to developers showing how to improve the components. The substitutability approach has been implemented and validated in the ComFoRT reasoning framework, and we report encouraging results on an industrial benchmark. This is an extended version of a paper, Dynamic Component Substitutability Analysis, published in the Proceedings of the Formal Methods 2005 Conference, Lecture Notes in Computer Science, vol. 3582, by the same authors. This research was sponsored by the National Science Foundation under grant nos. CNS-0411152, CCF-0429120, CCR-0121547, and CCR-0098072, the Semiconductor Research Corporation under grant no. TJ-1366, the US Army Research Office under grant no. DAAD19-01-1-0485, the Office of Naval Research under grant no. N00014-01-1-0796, the ICAST project and the Predictable Assembly from Certifiable Components (PACC) initiative at the Software Engineering Institute, Carnegie Mellon University. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the US government or any other entity.     

Combining Local and Global Model Checking

The verification process of reactive systems in local model checking [1,7] and in explicit state model checking is[13,15] on-the-fly. Therefore only those states of a system have to be traversed that are necessary to prove a property. In addition, if the property does not hold, than often only a small subset of the state space has to be traversed to produce a counterexample. Global model checking [6,23] and, in particular, symbolic model checking [4,22] can utilize compact representations of the state space, e.g. BDDs [3], to handle much larger designs than what is possible with local and explicit model checking. We present a new model checking algorithm for LTL that combines both approaches. In essence, it is a generalization of the tableau construction of [1] that enables the use of BDDs but still is on-the-fly.     

Analysis and verification of real-time systems using quantitative symbolic algorithms

The task of checking if a computer system satisfies its timing specifications is extremely important. These systems are often used in critical applications where failure to meet a deadline can have serious or even fatal consequences. This paper presents an efficient method for performing this verification task. In the proposed method a real-time system is modeled by a state-transition graph represented by binary decision diagrams. Efficient symbolic algorithms exhaustively explore the state space to determine whether the system satisfies a given specification. In addition, our approach computes quantitative timing information such as minimum and maximum time delays between given events. These results provide insight into the behavior of the system and assist in the determination of its temporal correctness. The technique evaluates how well the system works or how seriously it fails, as opposed to only whether it works or not. Based on these techniques a verification tool called Verus has been constructed. It has been used in the verification of several industrial real-time systems such as the robotics system described below. This demonstrates that the method proposed is efficient enough to be used in real-world designs. The examples verified show how the information produced can assist in designing more efficient and reliable real-time systems.