Subjective and objective evaluation of PBL outcomes in preclinical medical students.
Problem based learning curriculum is widely recognized as a
progressive, learner-centered, active learning approach and is currently
used in the entire medical curriculum in over 10% of medical schools
worldwide. But, is there real evidence that PBL is more effective than
traditional approaches? In this report, outcomes of a PBL tutorial in 55
second year MBBS students have been evaluated by the facilitators
(subjective evaluation) and by asking the students a set of questions
based on the intended outcome of the PBL (objective evaluation) soon
after the completion of PBL tutorial. In the subjective assessment by
the facilitators, all the students scored over 80% marks. In the
objective assessment, out of 55 students, only three students scored
over 50% marks. Perusal of answers to individual questions revealed
appalling lack of knowledge of the subject. To conclude, before
introduction of PBL-based curriculum in medical schools, usefulness of
PBLs in preclinical medical education needs to be thoroughly
investigated by objective evaluation of intended outcomes.
KEY WORDS: Medical education; Preclinical students; PBLs; Objective evaluation; Questionnaire
Medical students (Education)
|Author:||Marya, Ramesh Kumar|
|Publication:||Name: Internet Journal of Medical Update Publisher: Dr. Arun Kumar Agnihotri Audience: Academic; Professional Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2012 Dr. Arun Kumar Agnihotri ISSN: 1694-0423|
|Issue:||Date: Jan, 2012 Source Volume: 7 Source Issue: 1|
|Topic:||Event Code: 360 Services information|
|Geographic:||Geographic Scope: Malaysia Geographic Code: 9MALA Malaysia|
Problem-based learning (PBL) in medical education uses clinical cases as the context for students to study basic and clinical sciences. Problem based learning curriculum is widely recognized as a progressive, learner-centered, active learning approach and is currently used in the entire medical curriculum in over 10% of medical schools worldwide. (1) The advantages of PBLs over didactic lectures are said to be: (i) promotes deeper learning, (ii) self-directed learning-centered, (iii) enjoyable for students and teachers, (iv) improves team working skills, (v) facilitates lifelong learning, (vi) promotes horizontal and vertical integration of curriculum and (vii) relevant for future medical practice. In many medical schools with traditional curriculum, some PBLs are also held to give a "modern touch" to the curriculum. Is there real evidence that PBL is more effective than traditional approaches? All the so-called advantages of PBLs cannot be put to objective assessment. These advantages could possibly be wishful thinking on the part of advocates of PBL system of education. (2) In this report, outcomes of a PBL tutorial has been evaluated by the facilitators (subjective evaluation) and by asking the students a set of questions based on the intended outcome of the PBL (objective evaluation) soon after the completion of PBL tutorial.
The study was conducted in second year MBBS students (pre-clinical stage) in a medical school in Malaysia. Most of the students were Malaysian-Indians and a few were Malaysian-Chinese. The medical school follows a system-based integrated curriculum with horizontal and vertical integration. In each system, besides approximately 40 lectures, two PBLs are also held. This experiment was held during the study of renal system. The topic of one of the PBL tutorial was "acute glomerulonephritis (AGN)." Three sessions, each of three hour duration, were held on the topic. The students were given a set of learning objectives before the PBL tutorial (Table 1).
Assessment: During PBL sessions, each student was evaluated by the facilitators according to the following scheme:
* Participation/Cooperation etc (10 marks)
* Comprehension (10 marks)
* Reasoning/Application of knowledge (10 marks).
Soon after the end of PBL sessions, each student was asked to respond to a questionnaire (Table 2). The questions were not merely recall type. Most of the questions required comprehension/ reasoning/interpretation/application of knowledge.
In the subjective assessment by the facilitators, all the students scored between 25 and 29 out of 30 marks.
Marks based on their answers to the questionnaire (Max. marks: 40) are shown in Table 3.
Perusal of answers to individual questions revealed appalling lack of knowledge of the subject. Three students did not even know the color of normal urine: two said colorless, one said white. Sixteen students attributed the color of normal urine to urea and 12 students to bilirubin. Twenty-nine students did not know whether excretion of 800 ml urine per day is normal or abnormal. Twenty-seven students could not define oliguria. Forty students considered creatinine clearance 100 ml/min as being abnormal. Fifty-one students did not know the composition of red cell casts. None could draw a diagrammatic picture of a red cell cast. None of the students could define polyuria. Forty-seven students did not know the color of urine in obstructive jaundice. Only fifteen students mentioned the possible causes of death in a patient with acute glomerulonephritis. Only nine students could interpret the meaning of elevated antistreptolysin-O level. Twenty-four students considered less intake of water as a cause of oliguria. None could explain the mechanism of hypertension in cases with acute glomerulonephritis.
The evaluation of outcome of the PBL by subjective method by the facilitators revealed excellent results. All the students scored over 80 % marks. On the other hand, evaluation by objective questionnaire (Table 2) showed that the purpose of PBL has been totally defeated. Of 55 students only three students scored over 50% marks. Detailed study of the answers revealed that in spite of 9 hours spent in the tutorial, there was an amazing lack of basic understanding of renal physiology and pathophysiology of acute glomerulonephritis. PBLs are supposed to promote deeper learning. The author could not see any evidence of that. Whether the students enjoyed the exercise was not enquired into, since the students are paying huge fees in the medical school to gain knowledge to diagnose and treat a patient, not for enjoyment.
This exercise was given at the beginning of the renal course as a self-directed learning exercise. The analysis of the objective evaluation shows that most of the students learnt practically nothing. These results seems to lend credence to the widely held view that Asian medical students are unfit for PBLs. (3). Part of the problem of Asian medical students lies in the fact that they enter the medical school at an average age of 19 years as compared to 24 years in UK and 26 years in USA. At this age, Asian medical students lack self-confidence and critical thinking. They are keener to pass their examinations than to learn. This attitude arises from the fact that they are used to being spoon-fed throughout their years in school. They are not expected to have any views other than those of the teacher. In spite of all these facts, in the author's view, the poor results in PBL outcome shown by objective evaluation are primarily because of the introduction of PBLs in preclinical years. The students were given a clinical problem before they had the faintest idea about renal physiology and pathology. Imagine students given a PBL tutorial on acute abdomen even before they know the position of the various abdominal viscera. It is going to be a disaster but advocates of PBLs would never acknowledge it. Let preclinical year students in the Western world be subjected to such a PBL questionnaire as used in this study. It is likely that the results would not be any different. When faced with a similar situation, advocates of PBL shift their emphasis from PBL outcomes to processes. They claim that PBL provides a more challenging, motivating, and enjoyable approach to education as well as enhances the work environment for students and staff. (4) This argument may be valid for some courses other than medicine but in medical studies, outcome of a teaching method is of paramount importance.
To conclude, before introduction of PBL-based curriculum in medical schools, usefulness of PBLs in preclinical medical education needs to be thoroughly investigated by objective evaluation of intended outcomes.
(1.) Spencer JA, Jordan RG. Learner centered approaches in medical education. BMJ. 1999 May; 318(7193):1280-3.
(2.) Vernon DT, Blake RI. Does problem based learning work? A meta-analysis of evaluative research. Acad Med. 1993 Jul;68(7):550-63.
(3.) Kwan DCY. AMEA conference: Debate on PBLs. IMU, Kuala Lumpur, 2011, 5th March.
(4.) Schmidt HG. Problem based learning: rationale and description. Med Educ. 1983 Jan;17(1):11-6.
Ramesh Kumar Marya ([PSI] MD, PhD
Professor and Head, Unit of Physiology, AIMST University, Badong, Malaysia
(Received 25 April 2011 and accepted 01 May 2011)
[PSI] Correspondence at: Unit of Physiology, AIMST University, Badong, Malaysia; E-mail: firstname.lastname@example.org
Table 1: PBL on acute glomerulonephritis: learning objectives S.N. Learning objectives 1. Discuss the anatomy and development of the kidney. 2. What are various congenital anomalies of the kidney? 3. Discuss mechanism of urine formation. 4. Define GFR; what is its normal value and what are the factors affecting it? 5. List causes of oliguria. 6. How much is the normal renal blood flow and plasma flow? How is it calculated? 7. Describe the manifestations and complications of streptococcal infection? 8. What is the significance of age in relation to renal disease? 9. Describe the term clearance and its significance? 10. Discuss laboratory investigations of a urine sample. 11. Discuss the pathophysiology of edema with a special note on peri-orbital edema? 12. Discuss the pathophysiology of AGN 13. What are relevant investigations done for this case? 14. List the drug treatment for AGN. 15. What is the prognosis? 16. What is the differential diagnosis of this case? Table 2: Questionnaire: PBL on AGN Questionnaire: Fill in the blanks/tick off the correct S.N alternative answer. 2 marks for each question. 1. What is the color of normal urine?----Which chemical constituent produces this color?---- 2. A subject passes 800 ml urine/day. It is normal/abnormal.... 3. Define oliguria 4. Creatinine clearance 100 ml/min. It is normal/abnormal. 5. How does a patient suffering from hematuria describe his problem?. ... 6. What are the red cell casts composed of----? Draw the diagrammatic picture of a red cell cast. 7. Define polyuria 8. The color of urine in obstructive jaundice is 9. What can be the cause(s) of death in acute glomerulonephritis----, ----, ---- 10. Name two common clinical disorders causing hematuria----, ----. 11. In a patient of mild renal failure, when serum creatinine level is elevated, blood urea level may be normal. Is it possible?----, If yes, how?----. 12. A patient has elevated antistreptolysin O titer. What does it indicate?----. 13. Other than highly concentrated urine, what can be the cause of urinary specific gravity of 1.040? 14. Name two clinical conditions causing oliguria/anuria, even when kidneys are disease-free. ... 15. Besides kidney, which other organ(s) helps in regulation of blood pH. ... 16. In a patient, microscopic examination of urinary sediment revealed crystals of calcium oxalate. It is normal/abnormal. 17. Name the abnormal chemical constituents that may be present in urine----, ----, ----, ---- 18. In the presence of high concentration of ADH, the maximum percent of filtered water is reabsorbed in ---- segment of nephron. 19. Name two clinical tests for the evaluation of tubular function of the kidney----, ----. 20. In a patient with AGN, hypertension is due to----. Table 3: Answers to the questionnaire Marks Number of Students 0-5 1 6-10 12 11-15 27 16-20 12 21-25 2 >25 Nil Total 55
|Gale Copyright:||Copyright 2012 Gale, Cengage Learning. All rights reserved.|