Technology-integrated Assessment Scale

Colin Madland¹, Valerie Irvine¹, Christopher DeLuca², and Okan Bulut³

¹University of Victoria, ²Queens University, ³University of Alberta

2025-06-06

Developing the Technology-integrated Assessment Scale

Acknowledgements

Our team is spread across most of the country and we each live and work on land that has been cared for by Indigenous people for millenia. We each acknowledge that our abundant lives were made possible because of the displacement of the original stewards of the land.

https://cmad.land

• click ‘Slides’

Background - OTESSA24

Evolving our understanding of technology-integrated assessment: A review of the literature and development of a new framework

Two Papers

• Madland, C., Irvine, V., DeLuca, C., & Bulut, O. (2024b). Technology-Integrated Assessment: A Literature Review. The Open/Technology in Education, Society, and Scholarship Association Journal, 4(1), 1–48. https://doi.org/10.18357/otessaj.2024.4.1.57

• Madland, C., Irvine, V., DeLuca, C., & Bulut, O. (2024a). Developing the Technology-Integrated Assessment Framework. The Open/Technology in Education, Society, and Scholarship Association Journal, 4(1), 1–19. https://doi.org/10.18357/otessaj.2024.4.1.63

Technology-integrated Assessment Framework (TIAF)

4 Components

Assessment Design

Technology Acceptance

Duty of Care

Purposes of Assessment

Full Model

Alt-View

flowchart TD
classDef purpose fill:#440154,color:#fff,stroke:#440154,stroke-width:4px
classDef duty fill:#3B528B,color:#fff,stroke:#3B528B,stroke-width:4px
classDef accept fill:#21918c,color:#fff,stroke:#21918c,stroke-width:4px
classDef design fill:#5EC962,color:#000,stroke:#5EC962,stroke-width:4px
  A[Technology-Integrated Assessment] --- B(Assessment Purpose)
  A(Technology-Integrated Assessment) --- C(Duty of Care)
  A(Technology-Integrated Assessment) --- D(Technology Acceptance)
  A(Technology-Integrated Assessment) --- E(Assessment Design)
  B --- F([Assessment of Learning]) --- G([Assessment for Learning]) --- H([Assessment as Learning])
  C --- I([Bias]) --- J([Inclusion]) --- K([Relationships]) --- L([Ethical EdTech])
  D --- M([Performance Expectancy]) --- N([Effort Expectancy]) --- O([Social Influence]) --- P([Facilitating Conditions])
  E --- Q([Measurement Theory]) --- R([Academic Integrity]) --- S([Relevance]) --- T([Reciprocity])
  class B,F,G,H purpose
  class C,I,J,K,L duty
  class D,M,N,O,P accept
  class E,Q,R,S,T design

Differentiation

• 5Rs of Indigenous Education (Tessaro et al., 2018)
  • Relationship
  • Respect
    
  • Responsibility
  • Relevance
  • Reciprocity

Differentiation

• Duty of Care
  • Bias
  • Inclusion
  • Relationships
  • Ethical EdTech

Differentiation

• Technology Acceptance (Venkatesh et al., 2003)
  • Performance Expectancy
  • Effort Expectancy
  • Social Influence
  • Facilitating Conditions

Analysis

Exploratory Factor Analysis

• specify constructs
• identify parameters
• generate and refine items
• content validity

Table of Specifications

Parameters

Note: Archived /WIP/

Draft Questions

Pilot Questions

Content Validity

Expert Reviews

Next Steps

• pilot study @UVic
• scale revisions
  • 3 items per construct
• Survey administration acros BC PSE

Funding and Support

This  research  was supported by the  BCcampus  Research  Fellows  Program, which provides B.C. post-secondary educators and students with funding to conduct small-scale research on teaching and learning, as well as explore evidence-based teaching practices that focus on student success and learning.

References

Tessaro, D., Restoule, J.-P., Gaviria, P., Flessa, J., Lindeman, C., & Scully-Stewart, C. (2018). The Five R’s for Indigenizing Online Learning: A Case Study of the First Nations Schools’ Principals Course. Canadian Journal of Native Education, 40(1), 125–143.

Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425–478. https://doi.org/gc8zn2