End-of-course evaluations, conducted properly, can serve as valuable tools for improving online programs, but they’re not without their drawbacks.
“One of the problems is current students benefit little from the end-of-course surveys,” said Phil Ice, EdD, associate vice president of research and development at American Public University System. “Whenever you’re measuring what the student thinks of the course or their perceived learning, instructor performance, the way assets are utilized, you’re capturing that information retrospectively. So you’re not really helping the students who are engaged right now.”
Another problem relates to student demographics and learning needs, which can vary widely.
“If you’re a large campus, for instance, that has an on-campus cohort, an online cohort, and maybe a distance-based cohort, the end-of-course survey data can show differences between each of these populations that may call for the deployment of different design strategies, and different types of content for each of these three different cohort groups,” Ice said.
During the online video seminar, Data Driven Decision Making for Online Instructional Design, Ice explained how faculty and instructional designers can work together to better utilize the various and disparate data sets at their disposal to create a more complete picture of student engagement and learning.
Ice recommends using the Community of Inquiry (CoI) framework, which is grounded in a collaborative constructive view and examines the following independent but overlapping elements of the educational experience:
Social Presence – This is the degree to which students feel socially and emotionally connected to each other and to the instructor in a computer-mediated environment. Elements of social presence are demonstrated through affective expression, open communication and group cohesion. CoI questions that measure social presence include: “I felt comfortable conversing in the online medium” and “I felt that my point of view was acknowledged by other course participants.”
Teaching Presence – This involves the design, facilitation and direction of cognitive and social processes leading to personally meaningful and educationally worthwhile learning outcomes. Elements of teaching presence include setting curriculum and activities, shaping constructive discourse, and focusing and resolving issues. CoI questions that measure teaching presence include: “The instructor clearly communicated important course topics” and “The instructor helped to focus discussion on relevant issues in a way that helped me to learn.”
Cognitive Presence – The extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry. Elements of cognitive presence include triggering event (sense of puzzlement), exploration (sharing information and ideas), integration (connecting ideas), and resolution (synthesizing and applying new ideas). CoI questions that measure cognitive presence include: “Problems posed increased my interest in course issues” and “I have developed solutions to course problems that can be applied in practice.”