Research Article - International Journal of Medical Research & Health Sciences ( 2022) Volume 11, Issue 12
Preparing Blueprint for Theory Assessment of 3rd Part-I Students in Community Medicine
Jyoti Landge*, Nikhil Patel and Ruchita LunagariyaJyoti Landge, Department of Community Medicine, Sankalchand Patel University, Visnagar, Gujarat, India, Email: jlandge11@gmail.com
Received: 14-Jul-2022, Manuscript No. IJMRHS-22-69243; Editor assigned: 18-Jul-2022, Pre QC No. IJMRHS-22-69243 (PQ); Reviewed: 01-Aug-2022, QC No. IJMRHS-22-69243; Revised: 12-Oct-2022, Manuscript No. IJMRHS-22-69243 (R); Published: 21-Oct-2022
Abstract
A blueprint is a road map which specifies weightage to various topics in the whole curriculum. Blueprint helps to make assessment valid and reliable to some extent. We aimed to prepare a blueprint for assessment in the undergraduate community medicine curriculum as per latest modification in assessment in GMER 2019 and thereby test its utility. From study findings we found that blueprinting not only establishes a balance between teaching and learning but also improves validity and reliability. Thus, blueprinting for assessment is an asset for teaching learning process.
Keywords
Assessment, Blueprint, Community medicine, Reliability, Validity
Introduction
The term blueprint explains “a complete plan that explains how to do or develop something” [1]. In assessment, the term refers to a map or specification of assessment to ensure that all aspects of the curriculum and educational domains are covered by the assessment programme over a specified period of time [2]. It helps curriculum developers to match various competencies with the course content and the appropriate modality of assessment [3]. Assessment blueprint is an essential step for enhancing validity of assessment and constructive alignment particularly for high stakes examinations [4,5].
Recent study by Patil, et al. found that blueprinting helps to:
• Ensure questions being asked in examination are aligned to objectives.
• Ensure that there are no questions that are out of syllabus.
• Make assessment fair to the students.
• Design the instructional strategies.
• Ensure that the selected test items provide appropriate emphasis on thinking skills and assessment of in-depth knowledge.
• Sample content, competencies and tools for the assessment in a rational and balanced manner.
In most of medical colleges the traditional pattern of theory assessment tool of assessment is followed which may not follow the principles of assessment [6,7] the traditional pattern of theory assessment tool of assessment has many pitfalls and may not be uniform everywhere. Also literature found the qualitative feedback from students after any theory examination often suggests that question paper is not framed appropriately, not covered whole syllabus, missed the important topics, very lengthy and time was not adequate for writing the answers. Such type of scenario is common to almost all subjects but it is still more common for the subject of community medicine; owing to the vast nature of syllabus. Such types of errors in assessment results in a biased assessment. Change in curriculum (CBME) as per GMER 2019 i.e. traditional to CBA has emphasis on setting up an ongoing and longitudinal assessment so that teachers can identify the stage of the learner and decide whether they need further or different learning opportunities to acquire competency [8].
Blue printing is an effective assessment tool which can overcome existing problem in assessment, if not completely, to a large extent and hence make assessment more valid, so present study was carried out in Nootan Medical College and Research Centre (NMCRC), community medicine department.
Materials and Methods
A faculty development program was conducted in the department of community medicine to sensitize the faculty about the importance of blueprinting in assessment. Ethical committee approval was obtained. A blueprint was prepared for 3rd MBBS phase-1 (preliminary) written examinations (theory) in community medicine with inputs from all the faculty working above the rank of assistant professor and having minimum 3 years post graduate teaching experience (since this was the preliminary examinations as per pattern of university exam as per guidelines prescribed in GMER assessment module 2019, the complete syllabus was included in preparing a blueprint and assessment).
Division of topics
All the content areas of community medicine were listed in two parts of paper I and paper II according to university guidelines as follows.
Paper I include: Concept of health and disease, sociology, epidemiology, bio-statistics, communicable and noncommunicable diseases, environment health, nutrition, demographics and vital statistics and principles of health promotion and education.
Paper II includes: Demography and family planning, maternal and child health, health planning and management, occupational health, mental health, health education, health care delivery systems, national health programs, International health and voluntary health organization, disaster management and hospital waste management.
Skeleton of the assessment tool
As per the CBME pattern total allotted mark to theory community medicine is 200 marks. Thus, each paper is of 100 marks (108 marks if Optional questions are included). Each paper will have following sections (Table 1).
| Sr No | Question pattern | No. of questions | Marks | Marks if options are included |
|---|---|---|---|---|
| 1 | MCQs | 20 | 20 | 20 |
| 2 | LAQs | 3 | 30 | 30 |
| 3 | SAQs and SEQs | 12 out of 14 | 50 | 58 |
| Total | - | 37 | 100 | 108 |
| MCQs: Multiple Choice Questions; LAQs: Long Answer Questions; SAQs: Short Answer Questions; SEQs: Short Essay Question | ||||
Table 1: Skeleton of assessment tool.
Steps in preparations of blue prints
Decide the impact of each content area and allot an “Impact score” (I) to each areas: The Impact score (I) ranges from 1 to 3. Impact score 1 has less public health importance and having “nice to know” content areas for students. Impact score 2 has moderate public health importance and “desirable to know” content area and impact score 3 has high public health importance and “must know” content areas for students.
List the frequency of asking questions on that content areas and give “Frequency Score” (F) to each content area: Frequency score also ranges from 1 to 3. Frequency score 1 means less frequently asked question, frequency score 2 means moderate frequency of asking questions and frequency score 3 means high frequency of asking questions.
Decide weightage of the each content area (W): Following steps were conducted for deciding weightage to each content area.
• Calculate I × F i.e. Impact of topic × Frequency of asking questions from each topic.
• Calculate total summation of all I × F and this will be labeled as “T”.
• Weightage coefficient (W) will be calculated as I × F/T.
• Multiply the Weightage coefficient (W) by total no. of items i.e. 37. W × 37 will give us the proportionate weightage of each content area.
• Calculate adjusted weightage of each content area as per total marks i.e. 108.
Prepared blueprint was validated with the help of subject experts (faculty who had post of professor and completed Advance Course in Medical Education-ACME) and necessary changes were made accordingly. Feedback was taken from faculty.
Results
Final blue prints were prepared for community medicine after review and required correction (Tables 2-5).
| Sr No. | Topic paper-I | I | F | I*F | W=I*F/T | W*37 | Final marks (W*108/36) | ||
|---|---|---|---|---|---|---|---|---|---|
| 1 | Concept of health and disease | 2 | 2 | 4 | 0.07 | 2.7 | 3 | 9.0 | 9 |
| 2 | Relationship of social and behavioural to health and disease | 2 | 3 | 6 | 0.11 | 4.1 | 4 | 12.0 | 12 |
| 3 | Environmental health problems | 1 | 2 | 2 | 0.04 | 1.3 | 1 | 3.0 | 3 |
| 4 | Principles of health promotion and education | 3 | 2 | 6 | 0.11 | 4.1 | 4 | 12.0 | 12 |
| 5 | Nutrition | 3 | 2 | 6 | 0.11 | 4.1 | 4 | 12.0 | 12 |
| 6 | Basic statistics and its applications | 2 | 3 | 6 | 0.11 | 4.1 | 4 | 12.0 | 12 |
| 7 | Epidemiology | 3 | 3 | 9 | 0.17 | 6.1 | 6 | 18.0 | 18 |
| 8 | Epidemiology of communicable and non-communicable diseases | 3 | 3 | 9 | 0.17 | 6.1 | 6 | 18.0 | 18 |
| 9 | Demography and vital statistics | 3 | 2 | 6 | 0.11 | 4.1 | 4 | 12.0 | 12 |
| Total | 54 | 36 | 108 | ||||||
Table 2: Blueprint for paper-I topics.
| Topic paper-II | I | F | I*F | W=I*F/T | W*37 | Final marks (W*108/38) | ||
|---|---|---|---|---|---|---|---|---|
| Reproductive maternal and child health | 3 | 2 | 6 | 0.15 | 5.41 | 5 | 14.21 | 14 |
| Occupational Health | 2 | 2 | 4 | 0.10 | 3.61 | 4 | 11.37 | 12 |
| Geriatric services | 2 | 1 | 2 | 0.05 | 1.80 | 2 | 5.68 | 6 |
| Disaster Management | 2 | 2 | 4 | 0.10 | 3.61 | 4 | 11.37 | 11 |
| Hospital waste management | 1 | 2 | 2 | 0.05 | 1.80 | 2 | 5.68 | 6 |
| Mental Health | 2 | 1 | 2 | 0.05 | 1.80 | 2 | 5.68 | 6 |
| Health planning and management | 3 | 2 | 6 | 0.15 | 5.41 | 5 | 14.21 | 14 |
| Health care of the community | 3 | 2 | 6 | 0.15 | 5.41 | 5 | 14.21 | 14 |
| International Health | 2 | 2 | 4 | 0.10 | 3.61 | 4 | 11.37 | 11 |
| Essential Medicine | 2 | 1 | 2 | 0.05 | 1.80 | 2 | 5.68 | 6 |
| Recent advances in Community Medicine | 3 | 1 | 3 | 0.07 | 2.71 | 3 | 8.53 | 8 |
| Total | 41 | 38 | 108 | |||||
Table 3 :Blueprint for Paper-II topics.
| Topic paper-I | MCQ (1*20) | SAQ (3*6) | SEQ (5*8) | LAQ (10*3) | Total | Total Marks |
|---|---|---|---|---|---|---|
| Concept of health and disease | 3 | 2 | 5 | 9 | ||
| Relationship of social and behavioural to health and disease | 2 | 2 | 4 | 12 | ||
| Environmental health problems | 1 | 1 | 3 | |||
| Principles of health promotion and education | 4 | 1 | 1 | 6 | 12 | |
| Nutrition | 2 | 1 | 3 | 12 | ||
| Basic statistics and its applications | 1 | 2 | 1 | 4 | 12 | |
| Epidemiology | 3 | 1 | 1 | 5 | 18 | |
| Epidemiology of communicable and non-communicable diseases | 3 | 1 | 1 | 5 | 18 | |
| Demography and vital statistics | 2 | 2 | 4 | 12 | ||
| Total | 20 | 6 | 8 | 3 | 37 | 108 |
Table 4: Question distribution across paper-I topic.
| Topic paper-II | MCQ (1*20) | SAQ (3*6) | SEQ (5*8) | LAQ (10*3) | Total | Total Marks |
|---|---|---|---|---|---|---|
| Reproductive maternal and child health | 1 | 1 | 1 | 3 | 14 | |
| Occupational health | 2 | 1 | 3 | 12 | ||
| Geriatric services | 3 | 1 | 4 | 6 | ||
| Disaster management | 1 | 1 | 2 | 11 | ||
| Hospital waste management | 1 | 1 | 2 | 6 | ||
| Mental health | 1 | 1 | 2 | 6 | ||
| Health planning and management | 4 | 2 | 6 | 14 | ||
| Health care of the community | 1 | 1 | 2 | 4 | 14 | |
| International health | 3 | 1 | 1 | 5 | 11 | |
| Essential medicine | 3 | 1 | 4 | 6 | ||
| Recent advances in Community medicine | 1 | 1 | 2 | 8 | ||
| Total | 20 | 6 | 8 | 3 | 37 | 108 |
Table 5: Question distribution across paper-II topic.
Discussion
A blueprint serves as a map to define the content of a given assessment. Blueprint has its own advantages in curriculum implementation and assessment like weightage to various mapped competencies/topics; improve teaching planning and time management, a basis for sampling examination questions for systematic assessment of students. A blueprint of the mapped community medicine curriculum may thus not only outline the topic but also define their exact weightage to improve overall assessment of this vast subject in medical education. In this study, we aimed to prepare a blueprint for assessment in the 3rd MBBS community medicine curriculum as per the MCI’s/NMC‘s new competency based curriculum.
Earlier community medicine had two theory assessments paper of 60 marks each. With GMER 2019 notification and guideline in assessment module from 2019 admission batch onwards (year 2022 assessment in community medicine) each paper in community medicine will be of 100 marks. With this change in assessment pattern we had tried to make blueprint of community medicine preliminary examination which is reflective of university examination. Initially we tried to work on competency (k and KH) weightage for theory assessment but it was not feasible to allot questions on each competency as few competencies having weightage of less than 0.5 so we convert it into topic based weightage and final blueprint for paper I and II was prepared with the help of faculty members. After preparation it was sent for review.
Out of 10 reviewers that we had to validate our blueprint, 1 had differences regarding the I and F values allotted to certain topics. Suitable changes in the “I and F” were made wherever the authors felt that the suggested changes are relevant and acceptable.
As is obvious from findings, blueprinting will be a guide for paper setters to plan and construct questions from the eligible chapters. This not only reduced the paper setter’s selection bias but also improve the overall subject coverage and acceptability among students. Better topic coverage improves the validity and reliability of assessment. The elimination of construct over and under representation enabled us to better assess all topics based on its weightage.
The blue print makes the assessment clear, explicit and transparent to everyone involved in the process of learning. It makes assessment ‘fair’ to the students as they can have clear idea of what is being examined and can direct their learning efforts in that direction. Blueprints form the solid foundation for all systematic test development activity and provide evidence for the content related validity, thus making the assessment more meaningful. Findings of our study also agree with these advantages of learning mentioned earlier in literature.
Conclusion
Blueprinting improves validity, reliability and acceptability of both formative and summative assessments and thus establishes a balance between teaching and student’s overall learning. Blueprinting medical curriculum can be very helpful in actual execution of the CBME programme as implemented by the MCI/NMC.
Although the complete blueprinting is done as per the prescribed methodology, there is chance that allotment of “I and F” may vary from teacher to teacher/assessor to assessor/reviewer to reviewer.
Limitations
Student feedback was not taken as none of the batch was undergone through this pattern of exam.
Validity of prepared blueprint is subjective/individual view to allotment of “I and F.”
Acknowledgement
Faculty of community medicine and expert reviewer.
References
- Sood R, Paul V and Mittal S. Assessment in medical education: trends and tools. KL Wig CMET, India, 1995.
- Adkoli B and Deepak K. Blueprinting in assessment: principles of assessment in medical education, 1st ed. Jaypee Brothers Medical Publishers Ltd, India, 2012:205-213.
- Boland J, Finn Y and Geoghegan R. Blueprinting assessment to enhance constructive alignment. In: 16th Ottawa Conference, Transforming Healthcare through Excellence in Assessment and Evaluation. 25–29 April. Ontario, Canada, 2014.
- Patil SY, et al. Blueprinting in assessment: A tool to increase the validity of undergraduate written examinations in pathology. International Journal of Applied and Basic Medical Research, Vol. 5, 2015, pp. S76-S79. [Crossref][Googlescholar][Indexed]
- Sunita YP, et al. Blueprinting in assessment: How much is imprinted in our practice?. Journal of Educational Research and Medical Teacher, Vol. 291, 2014, pp. 4-6
- Gujarathi AP, et al. Preparation of blueprint for formative theory assessment of undergraduate medical students in community medicine. MVP Journal of Medical Sciences, 2015; Vol. 2, No. 2, 2015, pp. 100-103. [Crossref][Googlescholar]
- Medical Council of India. Assessment Module for Undergraduate Medical Education Training Program, 2019: pp 1-29.
- Downing SM and Haladyna TM. Validity and its threats. Assessment in Health Professions Education. New York, 2004;21-56.