Workshop - 3
Artificial Intelligence in Instrument Development
Introduction
Well-designed instruments are the backbone of high-quality healthcare research, ensuring valid, reliable, and contextually relevant measurements. AMEE Guide No. 87 provides a structured, seven-step framework for instrument development, and the integration of Artificial Intelligence (AI) can enhance each stage—from clarifying constructs and generating items to refining content, testing validity, and implementing adaptive tools. AI not only accelerates the development process but also improves precision and scalability, provided it is applied with methodological rigour and ethical safeguards. This workshop will equip participants with the knowledge and skills to integrate AI effectively into instrument development, ensuring tools are both scientifically sound and technologically advanced.
Objectives
By the end of this workshop, participants will be able to:
- Describe the AMEE Guide No. 87 framework for instrument development and its relevance in healthcare research.
- Identify AI applications that can strengthen each stage of instrument design, testing, and implementation.
- Apply AI-based tools to improve efficiency, accuracy, and scalability in developing healthcare instruments.
- Recognise and address ethical considerations, bias, and data privacy in AI-assisted research.
Target Audience
- Healthcare researchers
- Faculty members and medical educationists.
- Undergraduate medical students interested in research methodology.
- Postgraduate students and early-career researchers in healthcare disciplines.
Duration: 3 hours
| Session | Instrument Development Step | AI Application | Method | Duration |
| 1. Introduction & Context | Importance of rigorous instrument development and potential of AI. | Overview of AI platforms for healthcare research. | Lecture & Discussion | 15 min |
| 2. Literature Review | Identifying existing measures, conceptual frameworks, and gaps. | AI-powered literature search (e.g., Research Rabbit, Elicit, Semantic Scholar). | Live Demo | 20 min |
| 3. Interviews and/or Focus Groups | Gathering qualitative data from stakeholders. | AI-assisted transcription (e.g., Otter.ai, ChatGPT, Copilot, Gemini, NVivo, Perplexity) | Case Example & Demo | 20 min |
| 4. Synthesis of Literature Review & Interviews | Combining findings into a conceptual model. | AI-based clustering and concept mapping (e.g., ChatGPT, Atlis, NVivo etc). | Guided Group Exercise | 20 min |
| 5. Develop Items | Creating draft items/questions for the instrument. | AI-generated item banks, linguistic refinement, translation (e.g., Google Forms with AI plugins, SurveyMonkey Genius). | Hands-on Activity | 25 min |
| 6. Expert Validation | Seeking feedback from subject matter experts. | AI-assisted analysis of expert comments, thematic categorisation, and consensus scoring. (Expertise Finder, Google Forms, Tools like Consensus) | Interactive Exercise | 20 min |
| 7. Cognitive Interviews | Testing clarity and comprehension with small groups. | AI-based qualitative analysis of interview transcripts; sentiment and comprehension mapping. | Case Study | 20 min |
| 8. Pilot Testing | Trailing the instrument on a larger sample. | AI-enhanced data cleaning, reliability testing, and exploratory factor analysis (e.g., SPSS, R-based AI tools). | Hands-on Data Session | 25 min |
| 9. Ethical Considerations | Responsible and transparent use of AI in research. | Discussion of bias, data privacy, and responsible deployment. | Interactive Discussion | 15 min |
| 10. Wrap-up & Q&A | Summary of AI’s role across all steps; resources for further learning. | N/A | Open Forum | 10 min |
Dr. Sadia Fatima
MBBS, PGD, PhD (Glasgow)
Dr. Sadia Fatima is a medical doctor and human nutrition specialist with a PhD in Human Nutrition from the University of Glasgow, UK (2015) and an MBBS from Ayub Medical College, Abbottabad. She currently serves as Director of the Institute of Health Sciences Hazara, Khyber Medical University, and Associate Professor at the Institute of Basic Medical Sciences, Khyber Medical University, Peshawar.
Dr. Naveed Afzal Khan
MBBS, MHPE, PgDip. Family Medicine, HQ&PS, CHR
Dr. Naveed serves as the Director of the Quality Enhancement Cell and Assistant Professor of Medical Education at MTI Khyber Girls Medical College – Hayatabad Medical Complex. With a strong background in medical education, he is dedicated to advancing academic quality, curriculum development, and innovative teaching methodologies that align with international standards. His work focuses on enhancing institutional effectiveness, promoting evidence-based educational practices, and fostering continuous improvement to strengthen the learning environment for future healthcare professionals.
Dr. Naheed Mahsood
MBBS, MPH, MHPE, CHR, PhD Scholar(Medical Education)
Dr. Naheed is the Director of Research, Associate Professor, and Chairperson of the Department of Medical Education at MTI Khyber Girls Medical College – Hayatabad Medical Complex. With expertise in medical education, leadership, and research development, she plays a pivotal role in strengthening academic standards, fostering a research-driven culture, and guiding curriculum innovation. Her contributions focus on advancing evidence-based medical education, promoting faculty development, and enhancing institutional capacity to prepare future healthcare professionals for evolving global challenges.