Relationship Between TOEFL Score and Academic Success for International Engineering Students

Background Tests that classify English ability, such as the Test of English as a Foreign Language (TOEFL), are often the only application metric common to international applicants from a wide variety of academic backgrounds. As such, these test results are sometimes used beyond their intended scope to predict student academic success. Purpose (Hypothesis ) This study evaluated relationships between TOEFL scores and several measures of academic success for students at an American university abroad. Characterizing these relationships helps assess the scope of the TOEFL score's use in admissions decision making. Design /Method Linear and logistic regression were used to evaluate TOEFL score relative to overall grade point average (GPA), GPA for courses in engineering and in humanities, rate of passing a Comprehensive Assessment Examination (CAE), and graduation rate. High school GPA, gender, and nationality were also included as independent variables. Results A positive, statistically significant relationship was identified between TOEFL score and GPA, although weaker for engineering students than students in other fields, and for engineering courses than non‐engineering courses. TOEFL score was also statistically significant in logistic regressions of CAE pass rate and graduation rate, indicating increasing probability of success with increasing TOEFL score. However, model goodness‐of‐fit measures were relatively low, indicating many students whose performance defies general trends. Conclusions In spite of correlations between TOEFL score and academic performance, TOEFL scores should not be used in admissions beyond assessing individual students' English proficiency. Additional research is warranted to investigate trends that were identified related to gender effects and engineering‐specific student learning styles.     

An Examination of Indicators of Engineering Students' Success and Persistence

Student success and persistence within the major and university were examined through hierarchical linear and logistic regression analyses for two cohorts of engineering students. Indicators of success and persistence were based on theoretical and empirical evidence and included both cognitive and noncognitive variables. Cognitive variables included high school rank, SAT scores, and university cumulative grade point average. Noncognitive factors included academic motivation and institutional integration. Outcome variables included grade point average, enrollment at the university, and status as an engineering major. Gender differences also were evaluated. Several significant relationships among the variables were found. For instance, increased levels of motivation were significantly related to continuing in the major. Implications and directions for future research are discussed.     

The Effects of a First‐Year Engineering Design Course on Student Intellectual Development as Measured by the Perry Scheme

In response to the demand for enhanced design, problem‐solving, and team skills in engineering graduates, Penn State has instituted a number ofteam‐based, project‐learning courses, including one taken by nearly every first‐year engineering student. To determine the impact of these experiences on our students we have begun a cross‐sectional and longitudinal study of their intellectual development based upon the Perry model. In this paper, we describe the research methodology and results for the initial group of first‐year students interviewed. The results of the study include the effects on intellectual development of the first‐year design course, gender, honors status, and the students' academic ability as indicated by SAT scores and grade point average. Design experience was positively related to enhanced intellectual development. Honors status, gender, and academic ability were not significantly related to Perry rating. We discuss the implications of these findings for instruction and curricular reform.     

Is Modeling of Freshman Engineering Success Different from Modeling of Non‐Engineering Success?

The engineering community has recognized the need for a higher retention rate in freshman engineering. If we are to increase the freshman retention rate, we need to better understand the characteristics of academic success for engineering students. One approach is to compare academic performance of engineering students to that of non‐engineering students. This study explores the differences in predicting academic success (defined as the first year GPA) for freshman engineering students compared to three non‐engineering student sectors (Pre‐Med, STEM, and non‐STEM disciplines) within a university. Academic success is predicted with pre‐college variables from the UCLA/CIRP survey using factor analysis and regression analysis. Except for the factor related to the high school GPA and rank, the predictors for each student sector were discipline specific. Predictors unique to the engineering sector included the factors related to quantitative skills (ACT Math and Science test scores and placement test scores) and confidence in quantitative skills.     

A Method for Identifying Variables for Predicting STEM Enrollment

This research examines demographic, academic, attitudinal, and experiential data from the Cooperative Institutional Research Program (CIRP) for over 12,000 students at two universities to test a methodology for identifying variables showing significant differences between students intending to major in science, technology, engineering, or mathematics (STEM) versus non‐STEM subjects. The methodology utilizes basic statistical techniques to identify significant differences between STEM and non‐STEM students within seven population subgroups based upon school attended, race/ethnicity, and gender. The value of individual variables is assessed by how consistently significant differences are found across the subgroups. The variables found to be most valuable in identifying STEM students reflect both quantitative and qualitative measures. Quantitative measures of academic ability such as SAT mathematics score, high school grade point average, and to a lesser extent SAT verbal score are all indicators. Qualitative measures including self‐ratings of mathematical ability, computer skills, and academic ability are also good indicators.     

Measuring the Impact of an Individual Course on Students' Success

Background At the University of Michigan, qualified first‐year students who place out of the first‐semester calculus course may enroll in either the regular second‐semester calculus course or Applied Honors Calculus II. Students who enroll in Applied Honors Calculus II show higher academic performance than students enrolling in the Regular Calculus II. Purpose (Hypothesis ) The study addressed the question: does enrollment in Applied Honors Calculus II have a positive causal impact on subsequent academic performance for engineering students at the University of Michigan? Design /Method We acquired seven years of institutional data for engineering students who entered the University of Michigan from 1996 through 2003 and who qualified to enroll in Applied Honors Calculus II. Using regression analyses, we tested a causal model of impact of Applied Honors Calculus II on four measures of subsequent academic performance: grade in Physics II and average grade in all subsequent physics, mathematics, and engineering courses. Results After controlling for students' personal characteristics and prior academic achievement, the impact of Applied Honors Calculus II on students' academic performance was not statistically significant. In particular Advanced Placement scores accounted for the higher performance observed in Applied Honors Calculus II students. Conclusions We recommend including Advanced Placement scores in models that predict academic performance. Future research should also include measures of socioeconomic status (SES) and explore interactions between SES and academic background. Finally, in evaluations of specific curricula, the treatment effect—measured as treatment group mean minus control group mean, after controlling for covariates—is unlikely to be large if the control group receives high quality instruction.     

Influence of African American Engineering Student Perceptions of Campus Climate on Graduation Rates

African American undergraduate engineering student perceptions of institutional and personal/social campus climate factors were investigated to determine how these perceptions influence academic performance and institutional graduation rates. Data collection was accomplished through use of a quantitative and qualitative survey instrument administered to a national sample of subjects. The research discovered wide variation in individual institutional African American graduation rates and differences in rates among groups of institutions categorized by their academic selectivity or their designation as a Historically Black College and University. Students at institutions in the higher of the academic selectivity categories had higher graduation rates. However, students enrolled in the Historically Black Colleges and Universities category had more favorable perceptions of their college experience and had higher grades than students attending other institutions. After controlling for institution category, higher graduation rates were associated with students' lower perceptions of racism and discrimination and with students' greater institutional commitment.     

A Study of the Impact of Minority Engineering Programs at the FAMU‐FSU College of Engineering

The Engineering Concepts Institute, and the rest of the comprehensive minority student development program that followed it, has served students of Florida A&M University matriculating to the Florida A&M University—Florida State University (FAMU‐FSU) College of Engineering. A significant relationship between participation in the programs under study and graduation/retention was identified with the Pearson Chi‐squared test, Cochran‐Mantel‐Haenszel statistics, as well as the Mantel‐Haenszel estimate. Students who participated in the program were estimated to have significantly higher odds of five‐year graduation and six‐year graduation than students who did not participate in the program. A comparison of high school GPAs identified a selection bias, and high school GPA was then used to control for this selection bias in a multiple logistic regression model. While multiple individual cohorts remain statistically significant, the aggregate of all cohorts lacks significance due to the small number of participants and the possible overly stringent penalty imposed by the addition of high school GPA. Nevertheless, the results continue to show the positive trend observed earlier—that Minority Engineering Program participants are 25 percent more likely to be retained and graduate in engineering than students who had similar high school GPA but did not participate in the program—it is expected that continuing longitudinal study will bear out this trend as statistically significant. Although the programs described in this paper have been discontinued as a result of personnel changes, the study of the program is still useful as a contribution to the body of evidence supporting the effectiveness of such programs.     

An Analysis of Motivation Constructs with First‐Year Engineering Students: Relationships Among Expectancies,Values, Achievement,and Career Plans

Background Researchers have identified many factors affecting undergraduate engineering students' achievement and persistence. Yet, much of this research focuses on persistence within academia, with less attention to career plans after graduation. Furthermore, the relative influence of expectancy‐versus value‐related beliefs on students' achievement and career plans is not fully understood. Purpose (Hypothesis ) To address these gaps, we examined the relationships among the following motivation constructs for female and male first‐year engineering students: (a) expectancy‐related constructs that included engineering self‐efficacy (i.e., a judgment of one's ability to perform a task in engineering) and expectancy for success in engineering (i.e., the belief in the possibility of success in engineering); (b) value‐related constructs that included identification with engineering (i.e., the extent to which one defines the self through a role or performance in engineering) and engineering values (i.e., beliefs related to engineering interest, importance, and usefulness); (c) engineering achievement; and (d) engineering career plans. Design /Method Participants included 363 first‐year engineering students at a large state university. The students completed an online survey instrument in the first and second semester of their first year. Results Students' expectancy‐ and value‐related beliefs decreased over the first year for both men and women. Men reported higher levels for expectancy‐related beliefs than women. Expectancy‐related constructs predicted achievement better than the value‐related constructs, whereas value‐related constructs predicted career plans better for both men and women. Conclusions Expectancy‐ and value‐related constructs predicted different outcomes. Thus, both types of constructs are needed to understand students' achievement and career plans in engineering.     

Achieving Parity of the Sexes at the Undergraduate Level: A Study of Success

Most research about women in engineering focuses on reasons for their under‐representation. In contrast, we capitalized on an opportunity to study success: the School of Industrial Engineering at the University of Oklahoma had organically achieved parity of the sexes at the undergraduate level. To investigate this success, we adopted an ethnographic perspective, interviewing 185 students who represented four fields and four institutions as well as 12 faculty in Industrial Engineering at the University of Oklahoma. These data pointed to a combination of aspects of the discipline and the department culture as explanatory variables. Emerging from the data was a third explanatory variable: a high number of students, disproportionately many women, who relocated into Industrial Engineering from another major, underscoring the impact of broad recruiting activities. This paper emphasizes ideas that other departments can consider adapting to their own efforts to increase diversity.     

Undergraduate Student Competitions

This study explored student competitions for undergraduate engineering and engineering technology students to determine which institutions consistently win and what factors support their winning, and to obtain some insights into the benefits for students. Forty‐four student competitions for engineering and technology students were identified, and the first, second, and third place institutions from 2001 to 2003 were tabulated. Although one institution would often win a particular competition, no institution was a consistent winner for all competitions. Advisers of winning institutions reported that their institutions won consistently because of a dedicated faculty advisor and/or the close alignment of the competition with the institution's curriculum. Also important are a tradition of winning, the quality of the students, and (for hands‐on competitions) the availability of resources. Additional research is needed to determine if student competitions increase student learning.     

Women Engineering Students and Self‐Efficacy: A Multi‐Year,Multi‐Institution Study of Women Engineering Student Self‐Efficacy

As our nation's need for engineering professionals grows, educators and industry leaders are increasingly becoming concerned with how to attract women to this traditionally male career path. Self‐efficacy has been shown to be related to positive outcomes in studying and pursuing careers in non‐traditional fields. This paper describes the results of two years of engineering self‐efficacy data collected from women engineering students at five institutions across the U.S. This study adds to the growing body of self‐efficacy literature via its multi‐year, multi‐institution design and helps to clarify the impact of the engineering curriculum on self‐efficacy. Results indicate that while women students show positive progress on some self‐efficacy and related subscales, they show a significant decrease on feelings of inclusion from the first to second measurement period and further suggest a relationship between ethnicity and feelings of inclusion. Additionally, correlations show that self‐efficacy is related to women students' plans to persist in this predominantly male discipline.     

Project Preserve: A Program to Retain Minorities in Engineering

A national effort, begun in 1972 to increase the numbers of minorities in engineering, has made substantial progress, but the attrition for minority students is still estimated at 70%. To address the problem, Project Preserve was designed to confirm, by demonstration, the results of previous research showing that the combination of explicit cognitive development, close relationships with faculty, and strong bonds to an institution can raise the performance and retention of minority engineering students. The project enrolled over 100 minority engineering students who had been dismissed from freshman engineering studies on other campuses or placed on academic probation, but whose high school grades and SAT scores were indicative of success. They were channeled into participating institutions chosen for their record of basic support services, willingness to augment or restructure those services, and for variation among institutional variables: Xavier University of Louisiana (XU), California State University at Northridge (CSUN), and City College of the City University of New York (CCNY). A 2 1/2 hour battery of evaluation instruments was given to 79 students upon entry and to 26 students upon their attaining Junior status after two years in the program. The results demonstrated that: (1) an admirable 64% of would-be drop outs had achieved Junior status or were still enrolled in engineering at the conclusion of two years; (2) cognitive growth occurred on basic memory skills and one critical thinking skill, but there were institutional differences in the extent of intellectual development, with Xavier producing the most change; (3) closer relationships to faculty were outcomes; (4) the most positive feelings about, or bonding to, the institution occurred at Xavier University, while the most evidence of social participation and connectedness occurred at CSUN.     

Study Habits and Academic Achievement of Engineering Students

A study habits index measuring distractibility, inquisitiveness, and compulsiveness in test and homework situations was administered to 69 (27 women and 42 men) first-year, college engineering students. Scholastic Aptitude Test (SAT) scores and grade-point averages were also obtained. Two hypotheses were entertained: a) students who earn the highest grades, in contrast to less successful peers, will tend to be less distractible and more inquisitive; b) the advantage of such study habits will be evident when effects of academic aptitude (SAT performance) are controlled. Statistical tests supported both hypotheses. No significant sex differences were found for aptitude or grades, but women scored higher on the compulsiveness study habits scale than men. We argue that special programs to help some students study more meaningfully would likely improve overall academic performance.     

Gender and Ethnicity Differences in Freshmen Engineering Student Attitudes: A Cross‐Institutional Study*

We examine the attitudes of entering freshman engineering students and how they change over the course of the first year at 17 institutions. In addition to better understanding these attitudes and the changes that occur, we explore how these changes potentially affect such issues as “first term probation” and attrition from engineering programs. Particular attention is directed at isolating differences due to gender and ethnicity. Thirteen different student attitudes were captured using the Pittsburgh Freshman Engineering Attitude Survey© (PFEAS) at the beginning of the first semester (pre) and at either the end of the first semester or first academic year (post). Definite gender differences were found on the pre‐survey for five of the attitude measures. For all but one of these measures, female engineering students' initial attitudes were more negative than those of male students. Across the sample of institutions, female students consistently began their engineering studies with a lower confidence in background knowledge about engineering, their abilities to succeed in engineering, and their perceptions of how engineers contribute to society than did their male counterparts. However, those same female students were more comfortable with their study habits than were the male students. The post questionnaire data indicated that differences for three of these five attitude measures persisted. Most important, female engineering students continued to maintain a lower confidence in their abilities to succeed in engineering as compared to male engineering students. When the PFEAS data were mapped into EC 2000 outcomes, comparable cross‐institutional gender differences were observed that paralleled those found for the attitudinal measures. Because the number of minority students was relatively small, significant cross‐institutional differences between each minority cohort studied (African American, Asian Pacific, and Hispanic) and the majority cohort, similar cross‐institutional patterns could not be observed. However, possible trends were found between African American and majority students' attitudes for certain measures, while other attitudinal measures were found to be significant when Hispanic students were compared to majority students. Significant attitudinal differences between Asian Pacific and majority students were similar to those found between female and male engineering students. By knowing how attitudinal measures differ among gender and ethnic cohorts, and understanding how those differences relate to attrition from engineering programs, we can then developed more informed programmatic initiatives that can impact these attitude in a positive manner. As a result, we may be able to reduce engineering attrition, especially by underrepresented student cohorts.     

The Engage Program: Implementing and Assessing a New First Year Experience at the University of Tennessee

The Engage initiative at the University of Tennessee addresses the needs of entering engineering students through a new first year curriculum. The program integrates the engineering subject matter of the freshman year, teaches problem solving and design by application, and seeks to address the increased retention and graduation of engineering students. Noteworthy curriculum features of the Engage program include a hands‐on laboratory where students do physical homework to practice the concepts introduced in lectures, placing all freshman engineering students in a year‐long team design curriculum, and a team training course where engineering upperclassmen are trained in team facilitation techniques and placed as facilitators with the freshman design teams. The Engage program was piloted during the 1997–98 academic year with 60 students. In 1998–99, the program was scaled up to 150 students, and fully implemented with the entire freshman class of 465 students during the 1999–2000 academic year. Engage students have shown a significant increase in academic performance compared to students following a more traditional curriculum. Graduation statistics show the positive long‐term results of this effort.     

Considering Context: A Study of First‐Year Engineering Students

High‐quality engineering design requires an understanding of how the resulting engineered artifact interacts with society, the natural environment, and other aspects of context. This study examines how first‐year engineering undergraduates approached two engineering design tasks. We focused on how much students considered contextual factors during problem‐scoping, a critical part of the design process. As part of a larger, longitudinal study, we collected data from 160 students at four U.S. institutions. Students varied in their consideration of each design task's context, and women's responses were more likely to be context‐oriented than men's. Overall, context‐orientation was positively correlated between the two design tasks, despite differences in data collection and analysis. Having found that beginning engineering students, particularly women, are sensitive to important contextual factors, we suggest that efforts to broaden participation in engineering should consider legitimizing and fostering context‐oriented approaches to engineering earlier in the curriculum.     

Gender Differences in Elements of the Undergraduate Experience that Influence Satisfaction with the Engineering Major and the Intent to Pursue Engineering as a Career

Background Groups within and outside of educational institutions are interested in factors that influence satisfaction among students enrolled in the engineering major as well as elements within the college environment that shape students' intentions to work in engineering in the future and whether these elements differ by gender. Purpose (Hypothesis ) This study identified gender differences on indicators of the undergraduate experience including faculty‐related and student‐related variables as well as measures of satisfaction with the institutional environment that are related to satisfaction with the engineering major and intent to pursue a career in engineering ten years from now. Design /Method The mixed methods approach used for this investigation involved nine institutions with engineering undergraduate degree programs. An online questionnaire was administered to undergraduate students enrolled in engineering. Qualitative data was collected through focus group interviews with students at each of the nine participating institutions. Results Findings reveal that satisfaction with the engineering major does not translate directly to pursuing a career in engineering, particularly among women. In terms of elements of the undergraduate experience, some types of interactions with faculty and peers have both short‐ and long‐term impacts on interest in engineering as a major and as a career. Conclusions Creating learning environments that emphasize care and respect for students as well as overseeing student interaction during group work can make a difference in students' satisfaction in the engineering major and in interest in engineering as a career, particularly for women.     

A Paradigm for Assessing Conceptual and Procedural Knowledge in Engineering Students

Conceptual and procedural knowledge are two mutually‐supportive factors associated with the development of engineering skill. The present study extends previous work on undergraduate learning in engineering to provide further validation for an assessment paradigm capable of quantifying engineering students' conceptual and problem‐solving knowledge. Eight students who were enrolled in an introductory thermodynamics course and four who were enrolled in the course sequel provided verbal protocol data as they used instructional software. They were compared to existing data from a cohort of eleven science and engineering majors who had not taken thermodynamics. The results replicated earlier findings showing more cognitive activity on computer screens requiring overt user interaction compared to text‐based screens. The data also indicated that higher‐ versus lower‐performing students, based on course grades, engaged in more higher‐order cognitive processing. There was no evidence that students gained deeper cognitive processing as they advanced through the engineering curriculum.     

Self‐directed Learning Readiness Among Engineering Undergraduate Students

Two studies related to readiness for self‐directed learning of engineering students were performed using the Self‐directed Learning Readiness Scale (SDLRS). A cross‐sectional study of students in the first through final years of study showed that their SDLRS scores are significantly correlated with academic year of study and with grade point average, but not with gender. However, neither academic year of study nor grade point average is a good predictor of SDLRS scores; together they account for less than 5 percent of the observed variance. A second study investigated the effect of a problem‐based learning experience on students' readiness for self‐directed learning. It showed that the average readiness for self‐directed learning increased significantly for students in the problem‐based learning courses. However, investigation of the changes for individual students revealed that only nine of eighteen students showed significant increases in their SDLRS scores, and two showed significant decreases. Potential underlying causes are explored.