The Research Digest, February 2016, Issue 52

The Research Digest highlights new research in learning and teaching. This month, we are highlighting recent work on alignment and course learning outcomes. This issue was developed in conjunction with February’s issue of Best Practices on Assessing Learning Outcomes. If you’re interested in learning more about this topic, or have any suggestions for us, please send an email to lto@ryerson.ca. To access past issues of the Research Digest, visit the LTO website.

New research on the use of course learning outcomes:

Evidencing Learning Outcomes: A Multi-Level, Multi-Dimensional Course Alignment Model
Quality in Higher Education, 2015
This conceptual framework proposes a multi-level, multi-dimensional course alignment model to implement a contextualised constructive alignment of rubric design that authentically evidences and assesses learning outcomes. By embedding quality control mechanisms at each level for each dimension, this model facilitates the development of an aligned curriculum. The use of rubrics is a crucial theme for many higher education institutions owing to the binding requirement by universities to provide evidence to quality assurance agencies. The success of evidencing learning outcomes through rubrics, however, is only one piece of the puzzle. The other is the contextualised constructive alignment of intertwined factors. Despite the significance of embedding these factors, there has been little, if any, systematic framework in this area. The two key instrumental forces underpinning the conception of this model are: seeking external accreditation and the implementation of programme enhancement thus realising the strategic agenda for an Australian university.

Creating High Challenge/High Support Academic Environments through Constructive Alignment: Student Outcomes
Teaching in Higher Education, 2013
Higher education needs to provide challenging yet supportive learning environments catering for students with diverse academic needs. There is also an emphasis on using student-driven outcome measures to determine teaching effectiveness. How can these measures be used to reflect upon and evaluate teaching initiatives? Using an undergraduate occupational therapy programme as the site for exploration, this article reports on an application of constructive alignment principles and describes how available empirical data were used to explore student outcomes. A comparison was made between student evaluations and academic grades prior to, and after the implementation of the initiative. Results provide evidence of improvement in student satisfaction and academic grades as a result of implementing constructive alignment. Whilst it is acknowledged that changes in academic grades and student evaluations can be attributed to a number of factors, findings of this study support a view that constructive alignment facilitates students’ learning and experiences.

Comparison and Evaluation of Learning Outcomes from an International Perspective: Development of a Best-Practice Process
Journal of Chemical Education, 2015
Chemistry education focused on learning outcomes is increasingly practiced, providing new opportunities for international comparisons. The interest in intended learning outcomes and constructive alignment has grown in many parts of the world due to both research in higher education and political decisions. In an International Union of Pure and Applied Chemistry (IUPAC) project, we have developed a method of using critical evaluation of learning outcomes and descriptors at international, national, and institutional levels to enhance learner-centered chemistry education. This method is process-focused, aimed at learning by sharing and comparing practices around the world. Three overarching documents for the chemistry bachelor from the USA, Europe and Australia were compared. The differences were found to be more in style than in content. A tool for self-analysis was constructed to evaluate how learning outcomes for courses and modules are linked to each other and to learning outcomes for educational programs and how the expected learning outcomes can be aligned with learning activities and assessment. We conclude that the method can be used to elucidate the correspondence between learning outcomes at different levels, and the constructive alignment between learning outcomes, learning activities and assessment. The process gives new perspectives and shared knowledge. Chemistry education may need to be different depending on local considerations, and awareness of these differences is of value for further development.

A Report on the Implementation of the Blooming Biology Tool: Aligning Course Learning Outcomes with Assessments and Promoting Consistency in a Large Multi-Section First-Year Biology Course
Canadian Journal for the Scholarship of Teaching and Learning, 2010
The objectives of this study were to investigate the alignment of exam questions with course learning outcomes in a first year biology majors course, to examine gaps and overlaps in assessment of content amongst the sections of the course, and to use this information to provide feedback to the teaching team to further improve the course. Our ultimate goal was to provide students with learning outcomes that would clearly indicate the content and the level at which they would be expected to learn the content for this course, regardless of the section in which they were registered. We took an evidence-based approach to course evaluation and employed the Blooming Biology Tool to compare the learning outcomes and the exam questions of the course, investigating whether the cognitive skill level of each learning outcome as written matched the level at which it was assessed. We identified misalignments and recommended revising the learning outcomes to better reflect the intended level of learning for the course. We also investigated student performance on exam questions of different cognitive levels and found that students scored statistically significantly higher (p < 0.05) on questions in which learning outcomes were tested at the stated cognitive skill level compared to at a higher level.

Measuring Course Learning Outcomes
Journal of Learning Design, 2011
Accreditation criteria of programs require effective learning outcomes, assessment with documented procedures, tools, results, and actions to close the assessment loop with broad faculty involvement. This article describes a methodology for providing quantitative measurement of a course’s learning outcomes. The methodology uses a linkage matrix that associates each course learning outcome to one or more course assessment tool. The approach adopted provides a numeric score between 0 and 1 for each learning outcome with respect to each assessment tool and a combined score be calculated for each learning outcome from the tools associated with that outcome. The proposed methodology also provides insights into the consistency of the various assessment tools used to measure the achievement of a particular course learning outcome. The methodology described here has been successfully adopted in obtaining accreditation and reaccreditation to all programs offered by College of Computer Engineering at University of Tehran.

Assessment: Can It Lead to a Better Course Design?
Journal of College Science Teaching, 2013
Concerns about the misuse of assessment results in accountability related efforts led our biology faculty to take the initiative to develop our own assessment tool for one of our introductory biology courses. Our goal was to evaluate student achievement in the course and detect methods for improving our course. We developed a pre/posttest, which we used for two semesters. Results from these two semesters showed a very small but significant increase from pre to posttest scores. For the third semester of the study, we moved to a collegewide common final. By creating questions based on the learning outcomes of the course, we were able to compare scores for specific questions to learning outcomes. These comparisons provided insight into the difficulty of certain concepts in biology, along with the degree of uniformity of instruction across various sections of the course. These results led us to question the amount of information covered in the introductory course, the difficulty level of the course, and the vague nature of our learning outcomes. Presently, on the basis of the results, our biology faculty are in the process of reevaluating the content of the course and developing new, more detailed course learning outcomes.

This entry was posted in Research Digest. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *