Abstract
The rapid expansion of the healthcare research landscape requires medical students and healthcare professionals to master Information Literacy (IL) skills to effectively access and appraise evidence aligned with Evidence-Based Medicine, to make informed clinical decisions and improve patient care. At the Lee Kong Chian School of Medicine (LKCMedicine), Nanyang Technological University in Singapore, IL skills are embedded into the Years 1-3 Scientific Enquiry & Evidence-Based Medicine curriculum, taught collaboratively by a senior medical librarian and faculty. While traditional resources like PowerPoint slides and pre-recorded lectures provide theoretical knowledge, they lack interactive elements essential for engaging and ensuring IL skill retention needed for research.
By Year 4, students were observed struggling to recall and apply IL skills learnt earlier in the curriculum for their scientific research projects, often requiring IL skill refreshers extending to hour-long library consultations. Similarly, healthcare professionals engaged in medical research lack a flexible, "on-the-go" IL learning tool to accommodate their busy schedules. These challenges highlighted the need to develop an innovative digital learning solution that would complement the traditional resources.
Riley, an innovative 'rule-based' prototype chatbot, developed in 2023 for medical students via the Telegram app on smartphones, applied the DISCOVER framework for conversational agents. Riley featured four key IL skill chat modules guiding students on how to:
1. Use the PICO framework to form effective clinical questions.
2. Identify and use Medical Subject Headings (MeSH).
3. Form effective Boolean search strategies
4. Conduct a database search on PubMed to address a case scenario.
Each chat module included interactive elements like PubMed video demonstrations, comic narratives, mini-games, GIFs, and quizzes, with pause-and-check summaries to reinforce key concepts.
Riley was designed as a vibrant 3D avatar with dynamic expressions, using free online tools. Riley greets users by their Telegram usernames and kicks off interactions with ‘knock-knock’ jokes to capture their attention. Throughout the chat, Riley guides students with comic narratives that address research challenges on general health topics, making the learning experience both relatable and meaningful.
A pilot involving 55 students, randomly assigned, using pre-post survey questionnaires, revealed a higher satisfaction rate using Riley (Mean = 24, SD = 3.58) than traditional resources (Mean = 21.74, SD = 2.87). A significant interaction effect between Riley’s usage and time [F(1, 48) = 6.71, p = 0.01] suggested a positive impact on students’ IL learning. System Usability Test scores showed no significant difference [t(42.02) = 0.52, p = 0.6], suggesting both platforms were user-friendly, positioning Riley as a useful tool. Focus group interviews provided qualitative insights highlighting Riley as an innovative, engaging, and multimodal tool for IL learning.
Building on Riley's prototype success, a new library-faculty phase 2 team was formed in 2024 to upgrade Riley to Riley 2.0, incorporating adaptive learning paths, gamifications, and Gen-AI features. The team aims to expand data collection and introduce Riley 2.0 to healthcare professionals and patient advocates, broadening its impact across Singapore. Future updates will integrate user feedback and curriculum needs, ensuring Riley 2.0 remains both relevant and effective for IL and research.
References
1. Abdekhoda, M., & Dibaj, N. (2011). Information literacy skills among medical students: A case study at Tabriz University of Medical Sciences. Journal of Health Administration, 14(46), 65-72.
2. Bates, D. W., Saria, S., Ohno-Machado, L., Shah, A., & Escobar, G. (2022). Big data in healthcare: Using AI and analytics to improve outcomes. Journal of the American Medical Informatics Association, 29(4), 673-682.
3. Baskara, F. R. (2023). Chatbots and flipped learning: Enhancing student engagement and learning outcomes through personalized support and collaboration. IJORER: International Journal of Recent Educational Research, 4(2), 223–238. doi:10.46245/ijorer.v4i2.331
4. Bataineh, O., Qablan, A., Belbase, S., Takriti, R., & Tairab, H. (2022). Gender Disparity in Science,Technology, Engineering, and Mathematics (STEM) Programs at Jordanian Universities. Sustainability (Basel), 14(21), 14069. doi:10.3390/su142114069
5. Bullen M, Morgan T. Digital learners not digital natives. La Cuestión Universitaria. 2011;7:60–8.
6. Cook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., ... & Hamstra, S. J. (2013). Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA, 310(21), 2315-2327.
7. Cullen, R., Clark, M., & Esson, R. (2011). Evidence-based information-seeking skills of junior doctors entering the workforce: An evaluation of the impact of information literacy training during pre-clinical years. Health Information & Libraries Journal, 28(2), 119-129. https://doi.org/10.1111/j.1471-1842.2011.00934.x
8. Ellaway, R. H., Poulton, T., Fors, U., McGee, J. B., & Albright, S. (2021). Building a virtual patient commons. Medical Teacher, 43(1), 36-43.
9. Ellaway, R., & Masters, K. (2008). AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment. Medical Teacher, 30(5), 455-473.
10. Ghorashi, S. A., Zolfaghari, M., & Maghbouli, R. (2023). The Role of Advanced Technologies in Medical Education. Journal of Medical Education and Development, 15(2), 123-135.
11. Gross, M., & Latham, D. (2012). What’s skill got to do with it? Information literacy skills and self-views of ability among first-year college students. Journal of the American Society for Information Science and Technology, 63(3), 574-583. https://doi.org/10.1002/asi.21681
12. Hatcher-Martin, J. M., Adams, J. L., Anderson, E. R., Hood, K., Hung, C. S., Isaacson, S., ... & Schneider, R. B. (2021). Telemedicine in neurology: Telemedicine Work Group of the American Academy of Neurology update. Neurology, 94(1), 30-38.
13. Hege, I., Kononowicz, A. A., & Adler, M. (2020). A systematic review of models for assessing competence in ECG interpretation. BMC Medical Education, 20(1), 19.
14. Khalil, R., Tairab, H., Qablan, A., Alarabi, K., & Mansour, Y. (2023). STEM-Based Curriculum and Creative Thinking in High School Students. Education Sciences, 13(12), 1195. doi:10.3390/educsci13121195
15. Qablan, A., & Al-Qaderi, S. (2009). How to Change University Faculty Members’ Attitudes and Behavior in the Context of Education for Sustainable Development. Applied Environmental Education and Communication, 8(3-4), 184–194. doi:10.1080/15330150903269407
16. Wong, B. L. H., & Greenhalgh, T. (2021). Understanding digital health implementation: a case study of the NHS app in England. NPJ Digital Medicine, 4(1), 116.
