Outcomes Assessment of Engineering Writing at the University of Washington

Effective writing skills are crucial for engineers, and engineering programs have always struggled with how to prepare their students for the writing they will do as professionals. Now, programs must also show the Accreditation Board for Engineering and Technology (ABET) that they have clear educational outcomes for engineering communication and have a process for assessing student performance on those outcomes. At the University of Washington, we have spent the last five years developing an outcomes‐based assessment program for engineering writing. In spring 2001, the first round of writing assessment was completed. The assessment indicated that most of our students are competent in the outcomes we have developed. It also uncovered several weak areas, particularly in regard to working with sources and to adequately stating and supporting the purpose of the writing. We will be addressing these areas with additional instruction in the stand‐alone technical writing courses taken by engineering students. The process described in this paper could be helpful for other engineering programs preparing for ABET accreditation visits.     

Student Learning of Criterion 3 (a)‐(k) Outcomes with Short Instructional Modules and the Relationship to Bloom's Taxonomy

Engineering accreditation criteria require that engineering graduates demonstrate competency with a set of skills identified in Criterion 3 (a)‐(k). Because of a scarcity of instructional material on many of these topics, a team of engineering instructors developed and tested a set of short modules for teaching these skills. Using before and after module surveys, the students indicated their confidence in their ability to do specific tasks derived from the module's learning objectives. Data also were obtained with a control group not receiving the instruction. In comparing pre‐ and post‐module data, 33 percent of the comparisons were significantly different at the 0.05 level. In comparing control and post‐module data, the corresponding value was 44 percent. These results indicate that instruction with these short modules produced a significant effect on student learning.     

Turning Students into Professionals: Types of Knowledge and ABET Engineering Criteria

In order to implement the Accreditation Board for Engineering and Technology (ABET) engineering criteria, it is important for engineering educators to understand different types of knowledge and how these types relate to the outcomes described in Criterion 3. This paper first provides a heuristic for framing a program's educational objectives by identifying exemplars of the types of engineers a program seeks to graduate. Three sections then follow, each addressing categories of knowledge: tacit knowledge, four types of knowledge that can either be tacit or explicit, and knowledge created and shared in teams. Examples of the relationship of traits of knowledge to the outcomes in Criterion 3 are provided. The paper concludes with a brief discussion of how these types of knowledge are imbued in engineering students at the University of Virginia, as well as suggestions for additional ways in which they could be infused into engineering curricula.     

Use of Subject‐specific FE Exam Results in Outcomes Assessment

Engineering programs seeking accreditation are required by the Accreditation Board for Engineering and Technology (ABET) to document their continuous program improvement efforts and their outcomes. In this paper, we propose the subject‐specific results of the Fundamentals of Engineering (FE) Examination as one of the metrics suitable for assessing the outcomes of such efforts in individual courses. Statistical approaches appropriate for analyzing the FE Exam results and some caveats about their use are discussed. The application of this approach in the hydraulic engineering area of our program is illustrated.     

A Topical Analysis of Mechanical Engineering Curricula

This study has dissected the current curriculum in mechanical engineering into a list of required topics. The list indicates what material is currently considered to be the essential body of knowledge for graduating mechanical engineering students. It also provides a measure of the extent to which curricula differ from institution to institution. There is similarity in core material required among the institutions which we considered, but each one adds distinct requirements which give it an individual flavor or emphasis. The list reveals some of the differences among degree programs. While institutions have adjusted curricula to conform to the ABET engineering criteria, how they fulfill the “technical skill” outcomes is clearer than how they fulfill the “professional skill” outcomes. This survey shows that dissecting a degree program into required topics is useful for curriculum reform, as it provides a baseline to study the curriculum at a level more finely grained than a course.     

EC2000 and Measurement: How Much Precision is Enough?

As engineering faculty engage in the process of developing assessment plans to implement continuous quality improvement and satisfy the requirement of Engineering Criteria 2000 (EC2000), there is a concern about what measures are adequate to provide evidence that an engineering program is meeting its stated objectives. Some engineering programs are looking at using course grades as evidence that students are meeting the learning outcomes mandated by Criterion 3 of EC2000. After all, if there is a course in engineering design, why shouldn't grades be used to demonstrate that students are acquiring the skills necessary to meet the required outcome? The question remains as to whether or not course grades are adequate and/or efficient as a means to evaluate program effectiveness. This paper will define what is meant by educational inputs, processes, outputs, and outcomes in order to clarify the focus of the new “outcomes assessment” model of engineering education accreditation. A framework will be presented to clarify the meaning and scope of assessment activities needed to meet the information needs of academic programs and institutions. Models for course assessment and program assessment will be presented and the similarities and differences discussed.     

Comparison of Student Learning in Physical and Simulated Unit Operations Experiments

Industries are tending toward computer‐based simulation, monitoring, and control of processes. This trend suggests an opportunity to modernize engineering laboratory pedagogy to include computer experiments as well as tactile experiments. However, few studies report the impact of simulations upon student learning in engineering laboratories. We evaluated the impact of computer‐simulated experiments upon student learning in a senior unit operations laboratory. We compared data on control and test groups from three sources: 1) a comprehensive exam over the course; 2) a questionnaire answered by students regarding how well the areas of ABET Engineering Criterion 3 (a‐k) were met; and 3) oral presentations given by the students. Our results indicate that student learning is not adversely affected by introducing computer‐based experiments. We therefore conclude that, while the tactile laboratory should remain in the engineering curriculum, the pedagogy can reflect the increasing use of information technology in the manufacturing industries without compromising student learning.     

Teaching vs. Preaching: EC2000 and the Engineering Ethics Dilemma

The recently revised accreditation criteria issued by ABET have stirred renewed discussion of how and why to teach engineering ethics. This paper suggests that demonstrating students “understand ethics” need not (indeed, should not) imply that we assess whether our students “behave ethically,” either before or after graduation. Suggestions are provided for an approach focused on teaching ethics rather than preaching ethics, potential counter‐arguments are considered, and references to key resources in the engineering ethics literature are included.     

Industry Expectations of New Engineers: A Survey to Assist Curriculum Designers

In an era of unprecedented technological advancement, engineering practice continues to evolve but engineering education has not changed appreciably since the 1950s. This schism has prompted industry, government, and other key constituents to question the relevancy and efficacy of current programs. The Accreditation Board for Engineering and Technology (ABET) Engineering Criteria 2000, which will be fully implemented in 2001, emphasizes outcomes over process, and provides an opportunity for stakeholders to help universities define educational goals and objectives and design a curriculum to meet the desired outcomes.¹ While the need for curriculum reform has been acknowledged, the “industry position” was amorphous and anecdotal and therefore difficult to address. Qualitative methodologies such as formal surveys and structured interviews can be used to capture and quantify industry expectations of the needed attributes (i.e., knowledge, skills, and experience) for entry level engineering employees. Such instruments can provide key data useful in determining objectives and designing curricula to attain those objectives. This paper presents results of a formal survey of fifteen aerospace and defense companies concerning the perceived importance of 172 attributes related to the eleven ABET Program Outcomes and Assessment categories. The survey, resulting database, and preliminary analyses are available in hard copy and electronic form. This is the first formal survey and database resulting from efforts of the Industry-University-Government Roundtable for Enhancing Engineering Education (IUGREEE) to initiate a continuing, evolving process to provide curriculum designers with important information from industry.     

EC 2000: The Georgia Tech Experience

Engineering Criteria 2000 (EC 2000) is changing dramatically the manner in which engineering programs assess their curricula, interact with their constituents, and seek ABET accreditation. The College of Engineering at Georgia Tech served as a pilot program for the new engineering accreditation criteria. This paper shares some of Georgia Tech's experiences in preparing for EC 2000, some observations about the accreditation process, and offers seven suggestions for those preparing for an EC 2000 visit.     

The Use of Student Portfolios in Engineering Instruction

Student portfolios are listed as a possible means of assessment under the basic level accreditation criteria for ABET “Engineering Criteria 2000.” Efforts to initiate student portfolios in engineering instruction have been reported anecdotally in the literature, but a formal study on student portfolios in engineering has not been presented. We assigned student portfolios to freshmen and seniors taking biological engineering core courses containing a significant design component, and evaluated their effectiveness based on exit surveys, course evaluations, and instructor reflection. Portfolios were used to initiate student-centered learning, and to address the assessment issues raised by ABET. Results showed that 78% of seniors and 80% of freshmen believed that the use of student portfolios enhanced their learning. Among the freshmen, 69% of those students who were identified as sensing types by the Myers-Briggs Type Indicator found portfolios useful, while 87% of the intuitive types found them useful. In this paper, the methodologies for using portfolios are detailed, the results of applying the portfolio method are presented, and implications and recommendations are discussed.     

A Look at the Past and Present of General Engineering and Engineering Science Programs

In this article we discuss engineering programs named Engineering (sometimes referred to as General Engineering) and Engineering Science. Our purpose is to explore the role such non‐specialized programs have played, and currently play, in the overall scheme of engineering education. There are currently forty‐eight programs offered at U.S. institutions with EAC/ABET accreditation under the name Engineering or Engineering Science. Such programs are typically characterized by a general or interdisciplinary nature, and as such do not have to satisfy any discipline‐specific EAC/ABET program criteria beyond the basic criteria. Our analysis of Engineering and Engineering Science programs consists of two parts. First, we explore the historical trends in the evolution of such programs. Then we examine their uses, their current status, and their prospects for the future.     

Bridging Diverse Institutions,Multiple Engineering Departments,and Industry: A Case Study in Assessment Planning

As ABET and regional accreditation agencies focus their attention on student learning, effective planning processes and practices become critical to the departments and colleges that must initiate them. The lessons learned from the Synthesis Coalition's experience can be applied at the departmental level to meet the requirements of these agencies. The planning strategy focused on a collaborative process that involved all stakeholders: faculty, administrators, students, and industry representatives. The stakeholders validated and prioritized the Coalition's original goals and identified a specific set of abilities, criteria, and activities associated with them. (The goals included, to facilitate: communication, teamwork, hands-on facility with hardware, awareness of the social implications of engineering, modern industry practices, multi-disciplinary design, and open-ended problem solving.) The Coalition's industrial board provided written scenarios of “day in the life” challenges engineers face to help ensure that the values and goals the Coalition planned to assess matched those that industry felt were important. Next, course documents were analyzed to identify a set of assignments to use for developing authentic, performance-based assessment tools. This approach assured that by participating in the Synthesis Coalition assessment project, each campus prepared a core group of faculty to assess (and train other faculty to assess) student learning and ultimately document evidence of student learning for ABET accreditation.     

First Steps in Understanding Engineering Students' Growth of Conceptual and Procedural Knowledge in an Interactive Learning Context

The development of procedural knowledge in students, i.e., the ability to effectively solve domain problems, is the goal of many instructional initiatives in engineering education. The present study examined learning in a rich learning environment in which students read text, listened to narrations, interacted with simulations, and solved problems using instructional software for thermodynamics. Twenty‐three engineering and science majors who had not taken a thermodynamics course provided verbal protocol data as they used this software. The data were analyzed for cognitive processes. There were three major findings: (1) students expressed significantly more cognitive activity on computer screens requiring interaction compared to text‐based screens; (2) there were striking individual differences in the extent to which students employed the materials; and (3) verbalizations revealed that students applied predominantly lower‐level cognitive processes when engaging these materials, and they failed to connect the conceptual and procedural knowledge in ways that would lead to deeper understanding. The results provide a baseline for additional studies of more advanced students in order to gain insight into how students develop skill in engineering.     

Developing a Comprehensive Assessment Program for Engineering Education*

This paper provides an overview of one institution's efforts to establish a comprehensive assessment program for continuous improvement of engineering education. A five step systematic process to develop an integrated assessment program from identifying educational objectives to applying measurement methods is explained in detail. Activities to encourage faculty participation and commitment are outlined. Four integrated assessment processes used by both faculty and students to assess and provide performance feedback are described. The focus of these assessment methods is on the measurement, development, and improvement of student learning outcomes aligned with ABET Engineering Criteria 2000. Preliminary results and lessons learned from the overall experience are highlighted.     

Pre‐college Electrical Engineering Instruction: The Impact of Abstract vs. Contextualized Representation and Practice on Learning

Background While engineering instructional materials and practice problems for pre‐college students are often presented in the context of real‐life situations, college‐level texts are typically written in abstract form. Purpose (Hypothesis ) The goal of this study was to jointly examine the impact of contextualizing engineering instruction and varying the number of practice opportunities on pre‐college students' learning and learning perceptions. Design/ Method Using a 3 × 2 factorial design, students were randomly assigned to learn about electrical circuit analysis with an instructional program that represented problems in abstract, contextualized, or both forms, either with two practice problems or four practice problems. The abstract problems were devoid of any real‐life context and represented with standard abstract electrical circuit diagrams. The contextualized problems were anchored around real‐life scenarios and represented with life‐like images. The combined contextualized‐abstract condition added abstract circuit diagrams to the contextualized representation. To measure learning, students were given a problem‐solving near‐transfer post‐test. Learning perceptions were measured using a program‐rating survey where students had to rate the instructional program's diagrams, helpfulness, and difficulty. Results Students in the combined contextualized‐abstract condition scored higher on the post‐test, produced better problem representations, and rated the program's diagrams and helpfulness higher than their counterparts. Students who were given two practice problems gave higher program diagram and helpfulness ratings than those given four practice problems. Conclusions These findings suggest that pre‐college engineering instruction should consider anchoring learning in real‐life contexts and providing students with abstract problem representations that can be transferred to a variety of problems.     

Faculty Use and Impressions of Courseware Management Tools: A National Survey

Technology in the classroom is changing the way faculties instruct and students learn. Understanding how faculty members perceive and use technology for learning is important for improving the educational process because instructor perceptions can potentially be a hindrance to the use and implementation of technology. This paper describes the results of a survey that investigated faculty Internet usage for instructional purposes as well as their perceptions of courseware management and Web‐publishing tools. The survey targeted a random sample of engineering faculty at ABET‐accredited universities. The survey results show that while many faculty members are using both Web‐publishing tools and courseware management tools for delivering educational content, they use these tools for only a small subset of pedagogical activities.     

Preparing Students for Engineering Design & Practice

In 1995, the American Society of Civil Engineers sponsored an Education Conference to recommend changes for the Civil Engineering educational program. In addition, various other studies, such as that conducted by the American Society for Engineering Education (ASEE), investigated methods to strengthen undergraduate education. The present investigation of Lamar University students and alumni suggests that both practicing engineers and undergraduate/graduate students perceive that 2 design considerations or constraints presently have been and should be incorporated into the engineering design sequence at a high level. They include Engineering Codes and Standards, and Manufacturability (Constructability). In contrast, both students groups and practitioners rate two other design constraints: Social Ramifications, and Political Factors, at a relatively lower score. The foregoing design constraints are among those that have been adopted by the Accreditation Board for Engineering and Technology (ABET) as criteria that should be considered in the major design experience or course in order to satisfy the requirements of an accredited engineering degree.