Articles | Open Access |

A Quantitative Framework for Evaluating Technology-Enabled Active Learning (TEAL) Using Fleiss’ Kappa Inter-Rater Reliability Analysis

Arman Hossain , Sylhet Agricultural University
Zannatul Islam , Chittagong Veterinary and Animal Sciences University

Abstract

Technology-Enabled Active Learning (TEAL) has emerged as a transformative pedagogical approach that integrates digital tools with collaborative and inquiry-based learning strategies to enhance student engagement and knowledge construction. Despite its growing adoption, the evaluation of TEAL effectiveness remains fragmented and often lacks methodological rigor, particularly in assessing subjective learning outcomes across diverse evaluators. This study proposes a quantitative evaluation framework that integrates Fleiss’ Kappa inter-rater reliability analysis to systematically assess TEAL implementation and effectiveness. The framework operationalizes key constructs such as student engagement, collaborative learning dynamics, instructional design quality, and technological integration. By employing multi-rater assessment and statistical agreement measures, the proposed model ensures objectivity, consistency, and replicability in TEAL evaluation. The methodology is grounded in established educational theories and supported by empirical insights from existing literature. Findings indicate that incorporating inter-rater reliability enhances the validity of TEAL assessment and provides actionable insights for educators and policymakers. The study contributes a robust analytical model for educational evaluation and highlights the importance of structured assessment mechanisms in technology-integrated learning environments.

Keywords

Technology-Enabled Active Learning, Fleiss’ Kappa, Inter-Rater Reliability, Educational Evaluation Framework

References

Aesaert, K. et al.(2013). The content of educational technology curricula: a cross-curricular state of the art. Educational Technology Research and Development, 61(2), 131-151.

Aliyu, M.B., (2012). Integrating e-Learning in Technical and Vocational Education: A Technical Review. International Journal of Academic Research in Business and Social SciencesMay 2012, 2(5). ISSN: 2222-6990.

Bello, H., Shu’aibu, B., Saud, M.S., & Buntat, Y., (2013). ICT Skills for Technical and Vocational Education Graduates’ Employability. World Applied Sciences Journal,23(2). 204-207. ISSN 1818-4952. IDOSI Publications, 2013.

Bridgstock, R. (2009). The graduate attributes we’ve overlooked: Enhancing graduate employability through career management skills. Higher Education Research & Development,28(1), 31-44.

Chang, R. L., Stern, L., Sondergaard, H., & Hadgraft, R. (2009). Places for learning engineering: A preliminary report on informal learning spaces. In Proceedings of the Research in Engineering Education Symposium, Palm Cove, QLD. Retrieved December,12, 2009.

Che’ Ahmad, C.N., Osman, K. & Halim, L. (2010). Physical And Psychosocial Aspect S Of Science Laboratory Learning Environment. Procedia Social And Behavioral Sciences Journal, 9: 87-91.

Hanapi, Z., Nordin, M. S., & Rus, R. C. (2014). Unemployment Problem among Graduates of Technical Field: Competencies of the Graduates and Quality of the Education. Sains Humanika,2(2).

Hassan, N. F. (2015). Pemahaman pelajar melalui aplikasi TEAL (technology enabled active learning) dalam pembelajaran amali (Doctoral dissertation, Universiti Tun Hussein Onn Malaysia).

Hassan, N. F., & Puteh, S. (2017). A Survey of Technology Enabled Active Learning in Teaching and Learning Practices to Enhance the Quality of Engineering Students. Advanced Science Letters, 23(2), 1104-1108.

Hassan, N. F., Puteh, S., &Buhari, R. (2015). Student understanding through the application of technology enabled Active Learning in practical training. Procedia-Social and Behavioral Sciences, 204, 318-325.

Kirkup, L. (2015). Inquiry-oriented learning. Learning and Teaching Academic Standards project (LTAS). Australian Learning and teaching fellows. University of technology Sydney. GEDC, Adelaide, Dec. 2015.

Mouzakitis, G., (2010). The Role Of Vocational Education And Training Curricula In Economic Development. Procedia Social and Behavioral Sciences,2(4). 3914-3920.

Noordin, S., & Sapiee, F., (2010). Tahap Pencapaian Pengetahuan Pedagogi danKandungan Dalam Kalangan Bakal Guru Fizik.Repositori Universiti Teknologi Malaysia.

Park, E. L., & Choi, B. K. (2014). Transformation of classroom spaces: traditional versus active learning classroom in colleges.Higher Education,68(5), 749-771.

Pirker, J., Riffnaller Schiefer, M., Tomes, L. M., & Gutl, C. (2016). Motivational Active Learning in Blended and Virtual Learning Scenarios: Engaging Students in Digital Learning. Handbook of Research on Engaging Digital Natives in Higher Education Settings, 416.

Rocca, K. A. (2010). Student participation in the college classroom: An extended multidisciplinary literature review. Communication Education,59(2), 185–213.

Sanusi, A. M., & Puteh, S. (2017). An approach of excellence talent in engineering education programmed of enhancing the quality of students. Advanced Science Letters, 23(2), 1109-1112.

Saud,M.S., Shu'aibu, B., Yahaya. N., & Yasin, M.A.M., (2011). Effective Integration of Information and Communication Technologies (ICTs) in Technical And Vocational Education and Training (TVET) Toward Knowledge Management in The Changing World of Work. African Journal of Business Management,5(16). 6668-6673.

Stern, J., (2010). Introduction to Online Teaching and Learning, Laureate Education. Inc. Marina del Rey, CA.

Van Horne, S., Murniati, C.,& Saichaie, K. (2012). Assessing Teaching and Learning in Technology-Infused TILE Classrooms at the University of Iowa.in EDUCAUSE Learning Initiative’s Seeking Evidence of Impact.

Vaughan, N. D., Cleveland. I.M., & Garrison, D. R. (2013). Teaching in Blended Learning Environments: Creating and Sustaining Communitiesof Inquiry.Athabasca, AB: Athabasca University Press.

Article Statistics

Copyright License

Download Citations