E-collaboration within one supply chain and its impact on firms’ innovativeness and performance

The central premise of this paper is that e-collaboration plays a major role in achieving a sustainable competitive edge. In particular, we propose to examine the relative efficiency of electronic collaboration tools and to assess their impacts on the innovativeness and performance of individual firms positioned along a single supply chain. Empirical data from both the upstream and downstream perspectives for firms positioned at different points of one supply chain suggest that e-collaboration and its impacts create a one-sided benefit for the upstream side of the chain: the overall efficiency of e-collaboration tools is higher and the impacts of e-collaboration are more beneficial when used with suppliers than when used with customers. The results also point to a stage model for implementing collaboration tools in a supply chain: efficiency is higher for e-collaboration tools that support strongly that collaboration tools can have significant impacts on the supply chain and that these tools need to be implemented progressively, both upstream and downstream, thereby yielding different and, most probably, cumulative benefits over time. An earlier version of this paper was published in the Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS-36).     

Empirical evidence,evaluation criteria and challenges for the effectiveness of virtual and mixed reality tools for training operators of car service maintenance

The debate on effectiveness of virtual and mixed reality (VR/MR) tools for training professionals and operators is long-running with prominent contributions arguing that there are several shortfalls of experimental approaches and assessment criteria reported within the literature. In the automotive context, although car-makers were pioneers in the use of VR/MR tools for supporting designers, researchers started only recently to explore the effectiveness of VR/MR systems as mean for driving external operators of service centres to acquire the procedural skills necessary for car maintenance processes. In fact, from 463 journal articles on VR/MR tools for training published in the last thirty years, we identified only eight articles in which researchers experimentally tested the effectiveness of VR/MR tools for training service operators’ skills. To survey the current findings and the deficiencies of these eight studies, we use two main drivers: (i) a well-known framework of organizational training programmes, and (ii) a list of eleven evaluation criteria widely applied by researchers of different fields for assessing the effectiveness of training carried out with VR/MR systems. The analysis that we present allows us to: (i) identify a trend among automotive researchers of focusing their analysis only on car service operators’ performance in terms of time and errors, by leaving unexplored important pre- and post-training aspects that could affect the effectiveness of VR/MR tools to deliver training contents – e.g., people skills, previous experience, cibersickness, presence and engagement, usability and satisfaction and (ii) outline the future challenges for designing and assessing VR/MR tools for training car service operators.     

Evaluating the learning process of mechanical CAD students

There is little theoretical or experimental research on how beginner-level trainees learn CAD skills in formal training sessions. This work presents findings on how trainees develop their skills in utilizing a solid mechanical CAD tool (Pro/Engineer version 2000i² and later version Wildfire). Exercises at the beginner and intermediate levels were designed so that several variations of a solid object are built by non-experienced trainees as they accumulate training time. In this case, trainees are fourth year mechanical engineering seniors and as such, they were of a similar technical and gender make-up. This assessment was conducted over the duration of training (16-week long semester). The test exercises were used to assess the trainees’ speed and proficiency in the use of CAD by (1) measuring their performance time and (2) feature count (number of features-of-size used to build the test parts). Using performance time data, empirical learning curves are generated. Breaking these curves into declarative and procedural components provides insight into how fast the trainees develop cognitive and motor CAD skills. In order to confirm that this methodology can be extended to other CAD platforms, a follow-up study was performed on a different set of beginner-level trainees with similar make-up while using the same beginner-level parts but with a more recent version of Pro/Engineer: Wildfire. One significant result of this study is that the procedural and declarative components of CAD learning are largely cognitive.     

Advancing students’ computational thinking skills through educational robotics: A study on age and gender relevant differences

This work investigates the development of students’ computational thinking (CT) skills in the context of educational robotics (ER) learning activity. The study employs an appropriate CT model for operationalising and exploring students’ CT skills development in two different age groups (15 and 18 years old) and across gender. 164 students of different education levels (Junior high: 89; High vocational: 75) engaged in ER learning activities (2 hours per week, 11 weeks totally) and their CT skills were evaluated at different phases during the activity, using different modality (written and oral) assessment tools. The results suggest that: (a) students reach eventually the same level of CT skills development independent of their age and gender, (b) CT skills in most cases need time to fully develop (students’ scores improve significantly towards the end of the activity), (c) age and gender relevant differences appear when analysing students’ score in the various specific dimensions of the CT skills model, (d) the modality of the skill assessment instrument may have an impact on students’ performance, (e) girls appear in many situations to need more training time to reach the same skill level compared to boys.     

Supporting simultaneous versions for software evolution assessment

When reengineering software systems, maintainers should be able to assess and compare multiple change scenarios for a given goal, so as to choose the most pertinent one. Because they implicitly consider one single working copy, revision control systems do not scale up well to perform simultaneous analyses of multiple versions of systems. We designed Orion, an interactive prototyping tool for reengineering, to simulate changes and compare their impact on multiple versions of software source code models. Our approach offers an interactive simulation of changes, reuses existing assessment tools, and has the ability to hold multiple and branching versions simultaneously in memory. Specifically, we devise an infrastructure which optimizes memory usage of multiple versions for large models. This infrastructure uses an extension of the FAMIX source code meta-model but it is not limited to source code analysis tools since it can be applied to models in general. In this paper, we validate our approach by running benchmarks on memory usage and computation time of model queries on large models. Our benchmarks show that the Orion approach scales up well in terms of memory usage, while the current implementation could be optimized to lower its computation time. We also report on two large case studies on which we applied Orion.     

Can interactive visualization tools engage and support pre-university students in exploring non-trivial mathematical concepts?

Many students find it difficult to engage with mathematical concepts. As a relatively new class of learning tools, visualization tools may be able to promote higher levels of engagement with mathematical concepts. Often, development of new tools may outpace empirical evaluations of the effectiveness of these tools, especially in educational contexts. This seems to be the case with educational visualization tools. Much evidence about the effectiveness of these tools appears to be more suggestive than based on empirical evaluations. In this paper, we attempt to fill this gap and provide empirical evidence for the use of visualization tools in supporting exploratory and other learning-related activities. In particular, we aim to investigate whether visualization tools can be used to engage pre-university students in exploring non-trivial mathematical concepts. We focus particularly on this age group and content domain because of the difficulty these students may encounter when trying to investigate more challenging mathematical concepts. Also, it is during their formative years before university that students’ predisposition and likeness towards mathematical ideas are formed. We report in this paper a study assessing whether a visualization tool, whose design was informed explicitly by research from information visualization and human–computer interaction, could engage pre-university students in their exploration and learning of more advanced mathematical concepts. Students who participated in this study came from multiple grade levels and have diverse cognitive and language skills as well as preferences towards mathematics. The results of this study indicate that visualization tools can effectively engage these students and support their exploration of non-trivial mathematical concepts, only if the tool is designed such that it can cater the diverse needs of these students.     

Designing and assessing everyday objects: Impact of externalisation tools and judges’ backgrounds

Professional designers increasingly rely on IT-based systems in the course of their work. It is therefore crucial to know how such systems, especially computer-aided design (CAD) systems, influence both the design process and the final design, compared with the use of manual design methods. The objective of this paper is twofold: to address this issue and to suggest ways of improving IT-based systems specifically developed to support designers’ activities. In the first phase of a two-phase study, we investigated how the use of different tools can affect the design process. In the second phase, we conducted two complementary analyses to determine how the use of different tools affects the assessment of final designs, as well as the effect of the judges’ backgrounds on their assessments. We began by comparing the activities of two groups of designers: one group working with a CAD system, the other carrying out manual modelling. The results of this first phase revealed significant differences in the designers’ activities (changing the viewpoint, switching tools, correcting errors) according to the design method they used. We then asked judges from four different backgrounds (professional designers, design teachers, retailers and users), to assess the final designs according to specific assessment criteria (aesthetics, originality, functionality, marketability). The results of this second phase revealed a significant preference for objects designed with a CAD system. Furthermore, the judges attributed different levels of importance to each of the assessment criteria, depending on their background. This last result underscored differences in the mental models constructed by different groups of audiences involved in product development and marketing. In addition, it allowed us to identify their specific expectations about final designs. Based on the results of our analyses, we suggest ways of improving IT-based systems with a view to integrating design assessment more fully into CAD environments.     

Solution Generation with Qualitative Models of Preferences

We consider automated decision aids that help users select the best solution from a large set of options. For such tools to successfully accomplish their task, eliciting and representing users' decision preferences is a crucial task. It is usually too complex to get a complete and accurate model of their preferences, especially regarding the trade-offs between different criteria.We consider decision aid tools where users specify their preferences qualitatively : they are only able to state the criteria they consider, but not the precise numerical utility functions. For each criterion, the tool provides a standardized numerical function that is fixed and identical for all users and used to compare solutions. To compensate for the imprecision of this qualitative model, we let the user choose among a displayed set of possibilities rather than a single optimal solution. We consider the probability of finding the most preferred solution as a function of the number of displayed possibilities and the number of preferences. We present a probabilistic analysis, empirical validation on randomly generated configuration problems and a commercial application. We provide mathematical principles for the design of the selection mechanism, guaranteeing that users are able to find the target solution.     

Training for the job: evaluation of a self-defence training programme for correctional officers

Abstract

We investigated to what extent correctional officers were able to apply skills from their self-defence training in reality-based scenarios. Performance of nine self-defence skills were tested in different scenarios at three moments: before starting the self-defence training programme (Pre-test), halfway through (Post-test 1), and after (Post-test 2). Repeated measures analyses showed that performance on skills improved after the self-defence training. For each skill, however, there was a considerable number of correctional officers (range 4–73%) that showed insufficient performance on Post-test 2, indicating that after training they were not able to properly apply their skills in reality-based scenarios. Reality-based scenarios may be used to achieve fidelity in assessment of self-defence skills of correctional officers.

Practitioner summary: Self-defence training for correctional officers must be representative for the work field. By including reality-based scenarios in assessment, this study determined that correctional officers were not able to properly apply their learned skills in realistic contexts. Reality-based scenarios seem fit to detect discrepancies between training and the work field.

Abbreviations: DJI: Dutch National Agency for Correctional Insitutes; ICC: Intraclass Correlation Coefficient     

Application of machine learning techniques to the flexible assessment and improvement of requirements quality

It is already common to compute quantitative metrics of requirements to assess their quality. However, the risk is to build assessment methods and tools that are both arbitrary and rigid in the parameterization and combination of metrics. Specifically, we show that a linear combination of metrics is insufficient to adequately compute a global measure of quality. In this work, we propose to develop a flexible method to assess and improve the quality of requirements that can be adapted to different contexts, projects, organizations, and quality standards, with a high degree of automation. The domain experts contribute with an initial set of requirements that they have classified according to their quality, and we extract their quality metrics. We then use machine learning techniques to emulate the implicit expert’s quality function. We provide also a procedure to suggest improvements in bad requirements. We compare the obtained rule-based classifiers with different machine learning algorithms, obtaining measurements of effectiveness around 85%. We show as well the appearance of the generated rules and how to interpret them. The method is tailorable to different contexts, different styles to write requirements, and different demands in quality. The whole process of inferring and applying the quality rules adapted to each organization is highly automated.

     

What added value does a 1:1 student to laptop ratio bring to technology‐supported teaching and learning?

Abstract   The purpose of this study was to document typical use and configuration of 1:1 computing in two schools focusing on the added value and unique challenges these uses present. A qualitative case study design was used in two middle schools (sixth, seventh and eighth grade) in the south-eastern United States purposefully selected for their 1:1 computing programmes. Data were collected through formal and informal interviews, direct observations and site documents. Results indicated that online research, productivity tools, drill and practice, and eCommunications were the most frequent uses of computers in the 1:1 classroom. Moreover, the 1:1 classroom provided potentially transformative added value to these uses while simultaneously presenting unique management challenges to the teacher. In addition, the presence of 1:1 laptops did not automatically add value and their high financial costs underscore the need to provide teachers with high-quality professional development to ensure effective teaching. In order to create effective learning environments, teachers need opportunities to learn what instruction and assessment practices, curricular resources and classroom management skills work best in a 1:1 student to networked laptop classroom setting. Finally, researchers documented wide variation in fidelity to 1:1 computing, which suggests the need for further research exploring the conditions under which this variation exists.     

Prospective validation of a low-back disorder risk model and assessment of ergonomic interventions associated with manual materials handling tasks

The evaluation of low-back disorder risk associated with materials handling tasks can be performed using a variety of assessment tools. Most of these tools vary greatly in their underlying logic, yet few have been assessed for their predictive ability. It is important to document how well an assessment tool realistically reflects the job's injury risk, since only valid and accurate tools can reliably determine whether a given ergonomic intervention will result in a future reduction in back injuries. The goal of this study was to evaluate how well a previously reported low-back disorder (LBD) risk assessment model (Marras et al. 1993) could predict changes in LBD injury rates as the physical conditions to which employees are exposed were changed. Thirty-six repetitive materials handling jobs from 16 different companies were included in this prospective cohort study. Of these 36 jobs, 32 underwent an ergonomic intervention during the observation period, and four jobs in which no intervention occurred served as a comparison group. The trunk motions and workplace features of 142 employees performing these jobs were observed both before and after workplace interventions were incorporated. In addition, the jobs' LBD rates were documented for these pre- and post-intervention periods. The results indicated that a statistically significant correlation existed between changes in the jobs' estimated LBD risk values and changes in their actual low-back incidence rates over the observation period. Linear and Poisson regression models also were developed to predict a change in a job's incidence rate and the number of LBD on ajob respectively, as a function of the job's risk change using this assessment model. Finally, this prospective study showed which ergonomic interventions consistently reduced the jobs' mean low-back incidence rates. These results support use of the LBD risk model to assess accurately a job's potential to lead to low-back injuries among its employees.     

In-game assessment and training of nonverbal cognitive skills using TagTiles

We present a field study with a game for children called ‘Tap the little hedgehog’, which is played on the TagTiles console, a tangible electronic interface. The game was developed to train and assess cognitive skills and includes tasks which, in isolation, exhibit a high correlation with a number of subtests from the Wechsler Intelligence Scale for Children (WISC-III-NL). The tasks address a range of nonverbal skills by requiring children to perform different operations on abstract patterns such as copying, reproducing sequences from memory and mirroring patterns. In the current study, we tested whether these tasks kept their ability to address these skills if included in a gaming context, whether children are able to play the game independently and whether they are motivated to play the game. The results of the study support the hypothesis that nonverbal IQ-scores, as measured by the Wechsler Intelligence Scale for Children, can improve by training with a game. Hence, games like ‘Tap the little hedgehog’ can be used to train specific skills and serve as a screening tool for these skills. The results also confirm that children can play the game independently and that they enjoy it. We further found that children quickly learn how to play the game and use the interface.     

Vascular and nerve damage in workers exposed to vibrating tools. The importance of objective measurements of exposure time

The aim of the present study was to compare the development of vibration white fingers (VWF) in workers in relation to different ways of exposure estimation, and their relationship to the standard ISO 5349, annex A. Nineteen vibration exposed (grinding machines) male workers completed a questionnaire followed by a structured interview including questions regarding their estimated hand-held vibration exposure. Neurophysiological tests such as fractionated nerve conduction velocity in hands and arms, vibrotactile perception thresholds and temperature thresholds were determined. The subjective estimation of the mean daily exposure-time to vibrating tools was 192 min (range 18-480 min) among the workers. The estimated mean exposure time calculated from the consumption of grinding wheels was 42 min (range 18-60 min), approximately a four-fold overestimation (Wilcoxon's signed ranks test, p<0.001). Thus, objective measurements of the exposure time, related to the standard ISO 5349, which in this case were based on the consumption of grinding wheels, will in most cases give a better basis for adequate risk assessment than self-exposure assessment.     

The effect of competences on learning results an educational experience with a business simulator

Business games constitute a relevant e-learning method in management training. Their use and application has grown progressively, especially in business schools, universities and professional associations all over the world. The main reason for their popularity is that business games are seen as learning tools that allow students to acquire certain skills and competences above and beyond the skills fostered by other online or face-to-face methodologies. This study aims to analyse the relationship between generic competences and the learning results perceived by students in a business game. Analysis of the skills acquired using business simulators provides a wide field for potential future experimentation. The design of education plans is based entirely on the developing these skills. This study helps identify and assess which competencies are most linked to achieving the best learning outcomes, based on the value that students place on them, their satisfaction with them and their expectations of them. Only 16 of the 23 generic skills have a significant impact on achieving any of the objectives of the study variables.     

Comparison and evaluation of code clone detection techniques and tools: A qualitative approach

Over the last decade many techniques and tools for software clone detection have been proposed. In this paper, we provide a qualitative comparison and evaluation of the current state-of-the-art in clone detection techniques and tools, and organize the large amount of information into a coherent conceptual framework. We begin with background concepts, a generic clone detection process and an overall taxonomy of current techniques and tools. We then classify, compare and evaluate the techniques and tools in two different dimensions. First, we classify and compare approaches based on a number of facets, each of which has a set of (possibly overlapping) attributes. Second, we qualitatively evaluate the classified techniques and tools with respect to a taxonomy of editing scenarios designed to model the creation of Type-1, Type-2, Type-3 and Type-4 clones. Finally, we provide examples of how one might use the results of this study to choose the most appropriate clone detection tool or technique in the context of a particular set of goals and constraints. The primary contributions of this paper are: (1) a schema for classifying clone detection techniques and tools and a classification of current clone detectors based on this schema, and (2) a taxonomy of editing scenarios that produce different clone types and a qualitative evaluation of current clone detectors based on this taxonomy.     

Prospective validation of a low-back disorder risk model and assessment of ergonomic interventions associated with manual materials handling tasks

The evaluation of low-back disorder risk associated with materials handling tasks can be performed using a variety of assessment tools. Most of these tools vary greatly in their underlying logic, yet few have been assessed for their predictive ability. It is important to document how well an assessment tool realistically reflects the job's injury risk, since only valid and accurate tools can reliably determine whether a given ergonomic intervention will result in a future reduction in back injuries. The goal of this study was to evaluate how well a previously reported lowback disorder (LBD) risk assessment model (Marras et al. 1993) could predict changes in LBD injury rates as the physical conditions to which employees are exposed were changed. Thirty-six repetitive materials handling jobs from 16 different companies were included in this prospective cohort study. Of these 36 jobs, 32 underwent an ergonomic intervention during the observation period, and four jobs in which no intervention occurred served as a comparison group. The trunk motions and workplace features of 142 employees performing these jobs were observed both before and after workplace interventions were incorporated. In addition, the jobs' LBD rates were documented for these pre- and post-intervention periods. The results indicated that a statistically significant correlation existed between changes in the jobs' estimated LBD risk values and changes in their actual low-back incidence rates over the observation period.Linear and Poisson regression models also were developed to predict a change in a job's incidence rate and the number of LBD on a job respectively, as a function of the job's risk change using this assessment model. Finally, this prospective study showed which ergonomic interventions consistently reduced the jobs' mean low-back incidence rates. These results support use of the LBD risk model to assess accurately a job's potential to lead to low-back injuries among its employees.     

Learning to Apply Theory of Mind

In everyday life it is often important to have a mental model of the knowledge, beliefs, desires, and intentions of other people. Sometimes it is even useful to to have a correct model of their model of our own mental states: a second-order Theory of Mind. In order to investigate to what extent adults use and acquire complex skills and strategies in the domains of Theory of Mind and the related skill of natural language use, we conducted an experiment. It was based on a strategic game of imperfect information, in which it was beneficial for participants to have a good mental model of their opponent, and more specifically, to use second-order Theory of Mind. It was also beneficial for them to be aware of pragmatic inferences and of the possibility to choose between logical and pragmatic language use. We found that most participants did not seem to acquire these complex skills during the experiment when being exposed to the game for a number of different trials. Nevertheless, some participants did make use of advanced cognitive skills such as second-order Theory of Mind and appropriate choices between logical and pragmatic language use from the beginning. Thus, the results differ markedly from previous research.     

Exploring the main building blocks of SOA method: SOA maturity model perspective

The most successful enterprises plan its SOA adoption process as a series of maturity stages, where each stage acts as a foundation for the next. Various SOA maturity models (SOAMMs) are available to judge the current level of SOA adoption maturity of an enterprise. However, only a very few models review the formal use of SOA methods in the maturity assessment process. In this work, the SOAMMs are evaluated in terms of how they assess the methodological aspects of SOA adoption at each level of maturity. Based on methodical features, we introduce a set of building blocks for different maturity levels to assist the method engineers in SOA method definition. Welke’s SOAMM is selected as a base model to derive maturity guidelines for SOA methods. This work establishes the role of SOAMM in realizing the true potential of SOA and discusses why CMMI models are inapplicable to assess SOA adoption maturity. The main intention of this work is to develop a conceptual framework that describes the building block of SOA methods at different levels of maturity, including their processes, tools and technologies, and sourcing mechanisms. We adopt a real case study to determine the maturity level of SOA methodology dimension and to demonstrate how SOA adoption maturity can be improved by focusing on the methodical building blocks identified.     

A case study-based investigation of students’ experiences with social software tools

The term “social software” covers a range of tools which allow users to interact and share data with other users, primarily via the web. Blogs, wikis, podcasts and social networking websites are some of the tools that are being used in educational, social and business contexts. We have examined the use of social software in the UK further and higher education to collect evidence of the effective use of social software in student learning and engagement. We applied case study methodology involving educators and students from 26 initiatives. In this paper, we focus on the student experience: educational goals of using social software; benefits to the students; and the challenges they experience. Our investigations have shown that social software supports a variety of ways of learning: sharing of resources; collaborative learning; problem-based and inquiry-based learning; and reflective learning. Students gain transferable skills of team working, negotiation, communication and managing digital identities. Although these tools enhance a student's sense of community, the need to share and collaborate brings in additional responsibility and workload, which some students find inflexible and “forced”. Our findings show that students have concerns about usability, privacy and the public nature of social software tools for academic activities.