Assessment: Challenges

There are several challenges to sound learning assessment and program evaluation in engineering education programs.

We follow the challenges with examples of methods, tools, and forms developed by CDIO collaborators to assess student learning and evaluate programs. We include links to CDIO Web site papers and reports that describe the use and impact of some of these tools in engineering programs participating in the CDIO Initiative.

Challenges to sound learning assessment

1. Matching reliable, valid, and appropriate learning assessment methods and tools to all course learning objectives
Constructive alignment of learning assessment methods with course learning objectives presupposes that clear and relevant learning objectives have been specified for each course. This is not always the case in engineering programs. Moreover, appropriate learning assessment methods may not exist for some specific learning objectives.

2. Creating or adapting learning assessment tools that are appropriate, fair, and easily understood by both faculty and students
Engineering instructors are beginning to create and adapt appropriate learning assessment methods that align with their course learning objectives. One of the major goals of the CDIO Initiative is to support instructors and programs in the creation and implementation of new learning assessment methods.

3. Creating or adapting learning assessment methods that support deeper understanding of engineering concepts
Deeper understanding of engineering concepts is a pre-requisite for applying engineering knowledge and skills in practical and professional situations. Formative assessment methods, for example, the use of concept questions, give instructors and students opportunities to monitor the development of understanding and the surfacing of misconceptions. Summative assessment methods, for example, oral or written exams, need to target deep, conceptual understanding in addition to problem-solving applications.

4. Creating or adapting learning assessment methods that support active and experiential learning, and are suited to students' different learning styles
The CDIO Initiative promotes active and experiential learning, (See the Teaching and Learning section of the iKIT for more details on active learning.) Just as learning assessment methods are aligned with learning objectives, so too, are they aligned with teaching and learning methods. In some cases, the same method used to teach a concept or skill can be used for assessment, for example, concept questions or performance assessment. Using a variety of teaching, learning, and assessment methods makes it possible to accommodate a wider variety of learning styles among the students.

5. Committing time and resources to implement new learning assessment methods
Many engineering instructors, while expert in their disciplines, are either unaware of best practice in learning assessment, unsure of how to implement sound practices, or lacking time and resources required to develop and implement new approaches.

Addressing Challenges #1 through #5

• Methods and tools to assess conceptual understanding; and methods and tools to assess experiential learning, particularly in project-based courses, including communication skills and peer assessment. Download (.doc 84K)

Challenges to sound program evaluation

6. Implementing a variety of program evaluation methods to gather data from students, instructors, program leaders, alumni, and other key stakeholders that address the range of program goals
Similar to learning assessment methods, program evaluation methods need to be constructively aligned with program goals.  Sound program evaluation gathers data from all key stakeholder groups

7. Implementing and documenting a continuous improvement process based on program evaluation results
Most engineering programs collect volumes of data about their students, faculty, facilities, and stakeholders. The challenge is to analyze the data and summarize results into information that is useful for decision making. Program evaluation results need to form the basis for a continuous improvement process.

Addressing Challenges #6 and #7

• Methods and tools to evaluate courses, including student ratings and instructor reflections; methods and tools to evaluate students' overall experiences and satisfaction, including exit surveys and interviews; and methods and tools to evaluate students' overall achievement of program objectives. Download (.doc 48K)