Using Rasch measurement to investigate volleyball skills and inform coaching

This paper illustrates some of the ways that Rasch modeling techniques can be used to inform coaching of volleyball. Volleyball game statistics for 11 volleyball players collected over 27 matches on 3 skills were analyzed using a multifaceted Rasch model (Lincare, 1999) incorporating the partial credit model (Masters, 1982). Rating scale analyses and model fit statistics were used to derive interval level scales that provided more objective information regarding player ability and consistency than is usually available to coaches. Detailed results illustrate that Rasch analyses can provide very specific information that coaches might use in drill, practice, and game strategy design.     

Using Rasch analysis to construct a Clinical Problem-Solving Inventory in the dental clinic: a case study

A 28-item inventory was developed to measure the clinical problem-solving abilities of 3rd and 4th year dental students. The judgments of 57 expert raters (dental-school faculty) from four dental schools used the inventory to evaluate 183 dental students on a 5-point rating scale. The Rasch measurement model was employed to examine the psychometric properties and construct validity of this inventory. In this study, fit statistics identified the "noise" in the data and residual analysis assisted in extracting a meaningful structure. The study results indicate that the Rasch measurement model appeared to be a useful method for use in producing a unidimensional instrument. All five rating categories were used in a coherent manner, and four discernable levels of clinical problem-solving ability were identified. After removal of four repetitious items, a version of the Clinical Problem-Solving Inventory was finalized that could serve as a criterion measure for validating the use of a critical thinking test on the Dental Admission Test.     

Using Rasch measurement to investigate the cross-form equivalence and clinical utility of Spanish and English versions of a diabetes questionnaire: a pilot study

The purpose of this research was to use Rasch measurement to study the psychometric properties of data obtained from a newly developed Diabetes Questionnaire designed to measure diabetes knowledge, attitudes, and self-care. Specifically, a methodology using principles of Rasch measurement for investigating the cross-form equivalence of English and Spanish versions of the Diabetes Questionnaire was employed. A total of fifty diabetes patients responded to the questionnaire, with 26 participants completing the English version. Analyses detected problems with the attitude items. We attributed the scaling problems to the use of negatively worded items with participants having generally low educational backgrounds. Analysis of the knowledge and self-care items yielded unidimensional variables with clinically meaningful item hierarchies that may have relevance to treatment protocols. Furthermore, the knowledge and the self-care items from the two versions of the Diabetes Questionnaire met our criteria for establishing cross-form equivalence and thus allow quantitative comparisons of person measures across versions. Limitations of the study and suggested refinements of the Diabetes Questionnaire are discussed.     

Using the Rasch model to validate stages of understanding the energy concept

In recent years, there have been efforts to bridge science education with developmental psychology to develop theories on students developing understanding of science concepts from elementary to high school and beyond. The present study intends to test one such theory on students developing understanding of the energy concept. The theory states that students develop understanding of the energy concept by going through the following qualitatively distinct stages: (a) energy as activity/work; (b) energy as sources/forms, (c) energy transfer, (d) energy degradation, and (e) energy conservation. Three classes, one each from 4th grade, 8th grade, and high school physics class (grades 10, 11, and 12), completed a performance assessment. Students' performances were scored based on three traits of energy understanding: attention capacity, qualitative relations, and quantitative relations; each of the traits was defined into five hierarchical levels consistent with the five stages of understanding the energy concept. The Many-Facet Rasch Measurement (MFRM) model was used to analyze the effects of rater scoring severity, students' stages of energy understanding (theta), and difficulties of energy understanding traits. Results show that there was a discontinuity among the stages of understanding the energy concept, supporting the theory on students developing the understanding of the energy concept.     

Comparing screening approaches to investigate stability of common items in Rasch equating

This paper used real and simulated data sets to compare three screening approaches often used in state-wide equating programs utilizing the Rasch model: Wright and Stone's t-statistic, robust z-statistic, and displace. Analyses of real data sets supported the superiority of robust z-statistic and displace measure relative to Wright and Stone's t-statistic. The simulation component did not support the contention that indiscriminate use of the +/-0.3 logits criterion inflates rates of Type I error for robust z-statistic and displace measure, although this contention was supported for the Wright and Stone's t-statistic. However, Type II error rates were largest for displace measure, followed by the robust z-statistic, then the t-statistic. The paper discusses the importance of a priori selection of a criterion for screening linking items and its effects on stability and accuracy of Rasch equating constant.     

Measuring college sailing teams ability: an application of the many-facet Rasch model to ordinal data

For those who look at typical approaches to sports ranking, sailing seems an almost impossible challenge, thereby making the evidence supporting Rasch measurement in this study even more intriguing. This article describes our application of MFRM and the results from our analysis of one year's data from the North American college sailing competitions. We discuss the following issues for inclusion in the development of a Rasch model useful to college sailing team ability: 1. The level of data quality (as described by Stevens, 1946) 2. The connectedness of the contests 3. Empty cells (missing data) 4. Interpreting outliers, trends, or unusual results 5. Judges' or polls' bias Our results indicate the utility and precision of MFRM as a tool generally appropriate for ordinal ranking applications and sailing ability specifically.     

Using a multidimensional Rasch model approach to measure the police's perceived ability to detect,detain and intercept DWI vehicles when conducting sobriety checkpoints

This study developed a scale to measure 502 Taiwan traffic police officers’ perceived ability to detect, detain, and intercept those vehicles whose drivers are driving while intoxicated (DWI) when conducting sobriety checkpoints. Through factor analysis, the officers’ enforcement ability was found to consist of two component latent traits: detecting ability (DA) and detaining and intercepting ability (DIA). A multidimensional approach of Rasch models was then applied to measure the police officers’ perceived abilities and particular difficulties in conducting sobriety checkpoints. The study results indicated that the majority of police officers performed well in detecting DWI vehicles, but half of the study participants lacked confidence in detaining DWI vehicles and intercepting escaping DWI vehicles. DWI with weaving was found to be the most aggressive and threatening behavior to traffic police when conducting sobriety checkpoints. Police officers over age 46 were found to have significantly lower DA and DIA, while branch captains were found to have significantly higher DA than their colleagues. Several strategies and programs are suggested based on the study findings to improve the enforcement ability of police officers.     

Rasch analysis of a new construct: functional caregiving for adult children with intellectual disabilities

This research examined empirical evidence for a new construct, Functional Caregiving, which is a theory about mothers' caregiving of their adult children with intellectual disabilities. A sample of 108 biological mothers and primary caregivers rated survey items about their confidence to perform caregiving tasks. Rasch rating scale analysis found 61 items defined an empirical construct with three caregiving levels: Advocacy, Personal Caregiving, and Community. Results show item separation was 3.11 with high reliability, .91, and mother separation was 2.93 and reliability, .90. Both items and mothers showed adequate INFIT and OUTFIT values. Item invariance was confirmed between older and younger mothers, and principle components analysis of item residuals did not reveal any major dimensionality threats. Item decomposition analysis showed FC content theory to account for 58 percent of item calibration variance (R2 = .58, F = 42.3, p < .001). These results have important practical implications for health and social services, as well as family caregiving, interdisciplinary practices, and health policy development.     

Using Rasch modeling to measure acculturation in youth

Ethnic differences in health outcomes are assumed to reflect levels of acculturation, among other factors. Health surveys frequently include language and social interaction items taken from existing acculturation instruments. This study evaluated the dimensionality of responses to typical bilinear items in Latino youth using Rasch modeling. Two seven-item scales measuring Anglo-Hispanic orientation were adapted from Marin and Gamba (1996) and Cuellar, Arnold, and Maldonado (1995). Most of the items fit the Rasch model. However, there were gaps in both the Hispanic and Anglo scales. The Anglo items were not well targeted for the sample because most students reported they always spoke English. The lack of variability found in a heterogeneous sample of Latino youth has negative implications for the common practice of relying on language as a measure of acculturation. Acculturation instruments for youth probably need more sensitive items to discriminate linguistic differences, or to measure other factors.     

The Rasch model,additive conjoint measurement,and new models of probabilistic measurement theory

This research describes some of the similarities and differences between additive conjoint measurement (a type of fundamental measurement) and the Rasch model. It seems that there are many similarities between the two frameworks, however, their differences are nontrivial. For instance, while conjoint measurement specifies measurement scales using a data-free, non-numerical axiomatic frame of reference, the Rasch model specifies measurement scales using a numerical frame of reference that is, by definition, data dependent. In order to circumvent difficulties that can be realistically imposed by this data dependence, this research formalizes new non-parametric item response models. These models are probabilistic measurement theory models in the sense that they explicitly integrate the axiomatic ideas of measurement theory with the statistical ideas of order-restricted inference and Markov Chain Monte Carlo. The specifications of these models are rather flexible, as they can represent any one of several models used in psychometrics, such as Mokken's (1971) monotone homogeneity model, Scheiblechner's (1995) isotonic ordinal probabilistic model, or the Rasch (1960) model. The proposed non-parametric item response models are applied to analyze both real and simulated data sets.     

An approach to studying scale for students in higher education: a Rasch measurement model analysis

A questionnaire comprising 80 self-report items was designed to measure student Approaches to Studying in a higher education context. The items were conceptualized and designed from five learning orientations: a Deep Approach, a Surface Approach, a Strategic Approach, Clarity of Direction and Academic Self-Confidence, to include 40 attitude items and 40 corresponding behavior items. The study aimed to create a scale and investigate its psychometric properties using a Rasch measurement model. The convenience sample consisted of 350 students at an Australian university in 1998. The analysis supported the conceptual structure of the Scale as involving studying attitudes and behaviors towards five orientations to learning. Attitudes are mostly easier than behaviors, in line with the theory. Sixty-eight items fit the model and have good psychometric properties. The proportion of observed variance considered true is 92% and the Scale is well-targeted against the students. Some harder items are needed to improve the targeting and some further testing work needs to be done on the Surface Approach. In the Surface Approach and Clarity of Direction in Studying, attitudes make a lesser contribution than behaviors to the variable, Approaches to Studying.     

Understanding the unit in the Rasch model

The purpose of this paper is to explain the role of the unit implicit in the dichotomous Rasch model in determining the multiplicative factor of separation between measurements in a specified frame of reference. The explanation is provided at two complementary levels: first, in terms of the algebra of the model in which the role of an implicit, multiplicative constant is made explicit; and second, at a more fundamental level, in terms of the classical definition of measurement in the physical sciences. The Rasch model is characterized by statistical sufficiency, which arises from the requirement of invariant comparisons within a specified frame of reference. A frame of reference is defined by a class of persons responding to a class of items in a well-defined response context. The paper shows that two or more frames of reference may have different implicit units without destroying sufficiency. Understanding the role of the unit permits explication of the relationship between the Rasch model and the two parameter logistic model. The paper also summarises an approach that can be used in practice to express measurements across different frames of reference in the same unit.     

Rasch modeling and the measurement of social participation

Social participation is the main outcome of physical rehabilitation programs. The aim of this study is to improve the measurement of social participation, using an instrument called the Assessment of Life Habits Scale and the Rasch model. The interval level measurement, the dimensionality and the generalizability of the item hierarchy were verified. The data from a large sample of people with spinal cord injury was analyzed and specific results were compared with expert opinions. The main properties of the instrument were satisfactory and the agreement with expert opinion was high. Principal component analysis showed multidimensionality. The item difficulty hierarchy obtained with spinal cord injury experts was different from the one obtained with traumatic brain injury experts, indicating a different difficulty level of items in relation to each population characteristics. We conclude that the instrument is appropriate for the measurement of social participation and suggest ways to improve the instrument.     

Measuring teaching ability with the Rasch model by scaling a series of product and performance tasks

Rasch measurement can provide a much needed solution to scaling teacher ability. Typically, decisions about teacher ability are based on dichotomously scored certification tests focused on knowledge of content or pedagogy. This paper presents early developmental work of a partial-credit teacher-ability scale of 42 tasks (performances and products) with 348 rated items or criteria. The tasks and criteria are aligned with national and state standards for expected teacher knowledge and skills. These tasks are being used in about two-thirds of Florida school districts and are spreading to colleges of education. Over time there will be many variations in both tasks and criteria, but here we focus on the initial system and the Rasch model as part of the plan for development of the system.     

Using Rasch models to reveal contours of teachers' knowledge

Teachers' knowledge is usually categorised into subject matter (SMK) and pedagogical content knowledge (PCK). Previously, measurement instruments and consequent cognitive scales have been developed to assess students' and teachers' subject knowledge. A number of qualitative studies have explored teachers' pedagogical content knowledge. This study developed a means to investigate one aspect of PCK--teachers' awareness of their students' knowledge--using a combination of measurement and qualitative interpretation. We asked teachers to estimate on a Likert scale (and also describe qualitatively) the difficulty their pupils would have with test items which we had already scaled using data from their pupils. We then constructed, using various models, a "Teacher's collective Perception of Item Difficulty" (TPID) scale and contrasted this with the student's ability scale by comparing the two sets of item-difficulty parameters. The results were triangulated with qualitative data. We suggest the methodology is best supported by an Inverse Partial Credit Model (IPCM) but we compare the results across alternative Rasch models.     

Using the Rasch model to develop a measure of second language learners' willingness to communicate within a language classroom

The purpose of this investigation was to use Rasch measurement to study the psychometric properties of a 34 item questionnaire designed to measure second language learners' willingness to communicate (WTC) in English inside their language class. 490 Japanese university students' responses to the questionnaire were subjected to a number of different analyses. The first involved a comparison of the category threshold estimates produced by the Rating Scale and Partial Credit models. The questionnaire's items were then evaluated according to how well they defined the willingness to communicate construct. The potential dimensionality of using items that involved different speaking and writing tasks/situations in order to gain a more comprehensive understanding of students' willingness to communicate was also investigated. Next there was an examination of the questionnaire's four-point scale to ensure that it captured meaningful differences in students' WTC. Finally, the questionnaire items were compared using differential item functioning to determine if second year students were more willing than first year students in any of the different speaking and writing tasks/situations. This investigation closes with some suggestions on how the WTC questionnaire can inform second language instruction and curriculum design.     

Rasch measurement in the assessment of amytrophic lateral sclerosis patients

This paper examines the sensitivity to change over time of the Amyotrophic Lateral Sclerosis Assessment Questionnaire (ALSAQ-40). Individuals' health status change was assessed by means of the Rasch-based Reliable Change Index (RCI) for ALSAQ-40 questionnaires completed on two occasions, three months apart. In addition, at follow-up respondents indicated how much change they had experienced since baseline via dimension-specific self reported transition questions. 764 individuals returned questionnaires at baseline and follow-up. For all dimensions, of respondents defined by the RCI as worse, a majority rated themselves as worse. However, on two dimensions over 60% of the respondents who rated themselves as being worse were defined as unchanged by the RCI. As with effect size smaller RCI cut-off points might be needed for subjects with ALS. This study confirms that the ALSAQ-40 is a valid and responsive disease specific health related quality of life instrument for use in studies of patients with ALS or other motor neuron diseases.     

Rasch measurement in developing faculty ratings of students applying to graduate school

The Standardized Letter of Recommendation (SLR), a 28-item form, was created by ETS to supplement the qualitative rating of graduate school applicants' nonacademic qualities with a quantitative approach. The purpose of this study was to evaluate the following psychometric properties of the SLR using the Rasch rating scale model: dimensionality, reliability, item quality, and rating category effectiveness. Principal component and factor analyses were also conducted to examine the dimensionality of the SLR. Results revealed (a) two secondary factors underlay the data, along with a strong higher order factor, (b) item and person separation reliabilities were high, (c) noncognitive items tended to elicit higher endorsements than did cognitive items, and (d) a 5-point Likert scale functioned effectively. The psychometric properties of the SLR support the use of a composite score when reporting SLR scores and the utility of the SLR in higher education and in admissions.     

Rasch analysis of inattentive, hyperactive and impulsive behaviour in young children and the link with academic achievement

The Attention Deficit Hyperactivity Disorder (ADHD) criteria from the American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders were used to assess a large sample of children at the end of their first year at school in England. These data were explored using Rasch measurement and the measures for the items together with their frequencies are reported. The data were further analysed in three ways: a) The results were compared with a previous similar analysis of college students. b) A principal components analysis of the item residuals from the Rasch analysis was conducted. c) The measures were linked to reading and mathematics attainment assessed at three different time points. The exploration supported previous work and theoretical positions, and in doing so raised issues about the appropriateness of the use of the criteria across all ages. It also suggested that one of the currently recognised ADHD sub-types could be further sub-divided into verbal and physical hyperactivity. The links to academic achievement raised questions about the integrity of the currently recognised ADHD sub-types and the paper calls for further investigations.     

Measuring attitudes and behaviors to studying and learning for university students: a Rasch measurement model analysis

A Studying and Learning Scale was created using a model of Motivation (sets of ordered stem-items based on Striving for Excellence, Desire to Learn and Personal Incentives), with each item answered from three self-reported perspectives (an Ideal Self-view, a Capability Self-view, and a Studying and Learning Self-view). The response categories were the number of subjects studied. The stem-item sample was 23, each answered in three aspects, so each stem-item had three 'difficulties', making an effective item sample of 69. The person convenience sample was 372 students in education at an Australian university. The 69 items fit a Rasch measurement model and formed a scale in which the 'difficulties' of the items were ordered from 'easy' to 'hard' and the student measures of Studying and Learning were ordered from 'low' to 'high'. The person separation reliability was high at 0.94. The response categories were answered consistently and logically and the results supported many (but not all) of the conceptually ordered-by-difficulty item patterns. Students found it 'easy' to form a high view of How they would like to be, much 'harder' to form a high view of What they think they are capable of doing and even 'harder' to perform, at a high level, their Studying and Learning behavior for all stem-items, in accordance with the model.