Research Effectiveness
Research Statement
Educational assessment is at the heart of robust formal education systems and has been deeply impacted by both COVID-19 and by the broad emergence of generative artificial intelligence (genAI). My research program is focused, in the short-term, on quantitatively refining and operationalizing the TIAF in the context of emerging technologies, such as genAI and artificial/virtual reality (AR/VR). In the longer term, there is much work to be done to operationalize the refined TIAF by continuing to build a scale to be used to help individual educators, centres for teaching and learning, scholars of teaching and learning, and/or any other interested stakeholder to explore and understand the impacts of emerging technologies on assessment design and practice, and, by extension, to transform assessment practice in formal digital learning environments. The potential for transformative impact on education systems by paying attention to technology-integrated assessment is significant, as Ramsden claims, “assessment always defines the actual curriculum” (2003). I believe that transforming technology-integrated assessment practices is one of the most powerful levers we have in the drive to enable access to high quality higher education for all learners.
The TIAF comprises four primary constructs (assessment purpose, duty of care, technology acceptance, and measurement), each capable of supporting robust interdisciplinary quantitative, qualitative, and mixed-method investigations. Each of the four constructs is grounded in the 5 Rs of Indigenous education (respect, relevance, reciprocity, responsibility, and relationships) (Tessaro et al. 2018), making the TIAF a valuable tool for removing barriers to Indigenous learners and communities.
Questions that arise from the assessment purpose construct include investigating the impacts of AI use on the balance of assessment of/for/as learning in higher education practice. More specifically, emerging technologies including genAI and AR/VR, have provided opportunity for instructors to focus more on the process of learning and assessment for/as learning compared to the products of learning. The duty of care construct provides myriad opportunities for investigations into the ethics of genAI or AR/VR use and the ways in which it is important to prioritize human characteristics and relationships in education. Note especially the differential impacts on racialized learners and their challenges with using AI-powered remote proctoring services during the COVID-19 pandemic (see my proceeding that highlights my viral experience with this issue (Madland, Ofosuhene, and Adkins 2022)). At the same time, there are notable benefits for equity-deserving groups (e.g. neurodivergent learners) who may be able to leverage boutique, privacy-protecting genAI models to preview and refine their work prior to submission to an instructor. Duty of care is especially important to prioritize in Indigenous learning contexts where there is a long history of exploitation, extraction, and extermination in the context of education (see my proceeding based on exploring this issue (Madland and Restoule 2021)). The technology acceptance construct leads to questions about managing technological and pedagogical change in contexts with a focus on technology adoption by instructors. Lasting transformation of educational systems requires deep and careful work in managing change. My work as Director of Technology-integrated Learning and Assessment has given me clear insight and direct experience in managing and transforming educational systems and technological infrastructure at an enterprise level. This experience provides crucial context for my research into technology acceptance and adoption by the full breadth of stakeholders, from learners, to instructors, support staff, and senior administrators and board-level executives. Lastly, the assessment design construct offers many opportunities to consider pedagogical designs and architectures that maximize the impact of quality learning experiences. Too often, universal design for learning (UDL) is an afterthought or an add-on to existing educational and assessment designs, however in this framework, intentional focus is brought to bear on the up-front design of learning environments which are human-centred, equitable, inclusive, and accessible.
The TIAF as a whole and each individual construct described within present rich opportunities for research on learning through transformative digital systems. This wide variety of possible approaches to investigating emerging technologies in open, digital, and distance education leads to multiple opportunities for inclusive quantitative, qualitative, and mixed research collaborations for the benefit of all learners and particularly remote, rural, and Indigenous communities. Further, as the framework is embedded within the 5Rs of Indigenous education, there are multiple possibilities for collaborating with Indigenous scholars to explore these and other questions specific to Indigenous communities as we journey together towards reconciliation and renewal.
Funding
As a graduate of the Master of Education program at Athabasca University, I was honored with the Tim Byrne Memorial Scholarship for academic excellence and an outstanding thesis ($1000). I have also received funding from the Open Education Group ($5000 USD), the Thompson Rivers University Presentation Fund ($5000), two University of Victoria Graduate Student awards ($10,000 total), and a BCcampus Research Fellowship ($6000), all in support of my research.
To this point in my career, I have been completing my PhD part-time while engaged in a full-time career in higher education. This has left me with few options for finding funding (e.g, SSHRC funding is only recently available for part-time students, but I am no longer eligible because I am too far along in my program.) I look forward to being able to apply much more of my effort and energy into securing funding, not only through the Tri-Council agencies, but also through government, university, and other external sources. For example, I am privileged to have extensive background with the Technology Integration and Evaluation (TIE) Lab at the University of Victoria where I have been mentored in strategies for securing funding from agencies like the Canada Foundation for Innovation (CFI) which matches seed funding at a rate of 4:1. This would allow me to leverage start-up funds provided in this position and quadruple the financial impact of that initial funding. My supervisor and mentor used this strategy to raise $1.4 million ($780,000 from CFI, the largest amount at UVic) to support the development of the TIE Lab.
Knowledge Mobilization
As a rule, I endeavour to publish all my formal and informal research outputs under an open license and on the web. Below are two examples of these outputs. The first, and informal publication, emerged from my PhD coursework as a visiting student at the University of British Columbia. This publication is a critical family history, an in-depth exploration of one of my ancestors’ emigration from Norway to the USA, then to northern and central Alberta, and finally to British Columbia. The framework for this publication is the history of the treatment of Indigenous peoples throughout North America that opened the way for my great-grandfather to emigrate from Norway.
Madland, C. (2019). Critical Family History [Blog]. Colin M. Madland.
The second example of knowledge dissemination is related to a viral moment that I and a colleague experienced on Twitter (now defunct). This conference presentation and proceeding described our experiences with the facial detection algorithm on Zoom, which, when posted to Twitter, showed a related problem there.
Madland, C., Ofosuhene, M., & Adkins, J. (2022). Digital Platforms and Algorithmic Erasure: What are the Implications? OTESSA Conference 2022. OTESSA 2022, Online.
The original Tweet that set it all off is no longer available on X, but has been ported to BlueSky where it can be seen without the original context or replies.
Graduate Student Development
As a PhD candidate, I have not been in a position to mentor or supervise graduate students. That said, I am privileged to have been mentored by distinguished committee members from across Canada: Dr. Valerie Irvine (Supervisor, University of Victoria Director of the Technology Integration and Evaluation Research Lab and President of the Open/Technology in Education, Society, and Scholarship Association), Dr. Christopher DeLuca (Committee Member, Queen’s University Associate Dean of Graduate Studies), and Dr. Okan Bulut (Committee Member, Professor of Measurement, Evaluation and Data Science, University of Alberta and University of Alberta President’s Research Prize Recipient). Each of these mentors has modelled high standards of academic excellence comibined with both support and trust as appropriate. This blend of personal mentoring and adherence to academic excellence has given me a template from which I can build my own practice of mentoring graduate students.
In my role of Director of Technology-Integrated Learning and Assessment, I have the privilege of supervising multiple early career educational technologists, several of whom have since started their journeys through masters and doctoral programs. As such, I have had the opportunity to informally mentor these people as they know that I am nearing completion of my program and seek out my advice. I know from these less formal encounters that each graduate student brings their own experiences, strengths, challenges, and fears to their programs, necessitating a personalized approach for each one.