In the fall of 2021, I embarked on the BCcampus Student Research Fellowship, a project funded by BCcampus intended to promote research on teaching and learning in BC. My project aimed at technology-integrated assessment practices in higher education. Assessment is already a contentious and high-stakes topic in higher education; however, concern around assessment escalated due to the COVID-19 pandemic, and also in light of generative AI models. The fellowship provided funding and support for a component of my PhD dissertation research at the University of Victoria, where I am supervised by Dr. Valerie Irvine.
My research is ongoing, and I am pleased to report that I have hit a milestone in completing and publishing the first two of three papers that will form my dissertation. The first paper is a literature review exploring how higher education instructors integrate technology and assessment (Madland et al., 2024b). We purposely explored papers written by those who do not study assessment as a disciplinary area, but who are instructors in a wide variety of disciplines and who are compelled for one reason or another to use technology when they assess learning. We explored these articles through the lens of the assessment design in a digital world framework published by Bearman et al. (2022). While there were areas of overlap between the literature and the Bearman et al. framework (primarily the importance of understanding the different purposes of assessment), there were also themes that emerged in the literature that are not modeled in the framework (e.g., instructor workload, academic integrity), and there were components of the framework that were not evident in the literature (e.g., digital literacies). These points of incongruity suggested a need for revisions to the model which led to a process of envisioning what a model of technology-integrated assessment might look like.
The development of the technology-integrated assessment framework, described in the second paper (Madland et al., 2024a), is the result of this envisioning process and serves as a good anchor point in beginning to understand how to improve technology-integrated assessment practices in higher education in British Columbia and beyond. The technology-integrated assessment framework consists of four components, which might be considered to be four factors that instructors consider when planning assessment. The four factors and their sub-factors are:
- The purposes of assessment
- Assessment of Learning
- Assessment for Learning
- Assessment as Learning
- Duty of Care
- Bias
- Inclusion
- Relationships
- Ethical EdTech
- Technology Acceptance
- Performance Expectancy
- Effort Expectancy
- Social Influences
- Facilitating Conditions
- Assessment Design
- Measurement Theory
- Academic Integrity
- Relevance
- Reciprocity
Figure 1 is a theoretical conceptualization of the model and its sub-components.
Note: Licensed (CC-BY), from Madland, C., Irvine, V., DeLuca, C., & Bulut, O. (2024). Developing the Technology-Integrated Assessment Framework. The Open/Technology in Education, Society, and Scholarship Association Journal, 4(1), 1–19. https://doi.org/10.18357/otessaj.2024.4.1.63.
We were intentional about considering the framework in light of the 5Rs of Indigenous education: respect, relevance, reciprocity, responsibility, and relationships (Kirkness & Barnhardt, 1991; Tessaro et al., 2018). Respect and responsibility are embedded in the ‘Duty of Care’ component while relationships, reciprocity, and relevance are explicit in the framework. Further work is coming and will focus on exploring the relationships between the factors and sub-factors in the framework.
I am grateful to my supervisor, Dr. Valerie Irvine, for her keen insight and expert contributions, as well as my committee members, Dr. Chris DeLuca (Queen’s University), and Dr. Okan Bulut (University of Alberta) whose diverse wisdom around assessment were invaluable.
I am also appreciative to BCcampus, especially Leva Lee, Britt Dzioba, Rebecca Shortt, and Gwen Nguyen for their support through the BCcampus Student Fellowship program, which provides B.C. post-secondary educators and students with funding to conduct research on teaching and learning, as well as explore evidence-based teaching practices that focus on student success and learning.