This article describes the development by a trainer group of an instrument for assessing teaching skills. It comments on the benefits of developing and using the instrument and also some of the problems encountered. Finally it suggests ways in which the instrument can be further modified and improved.
A great tool for assessing your technique for doing Random Cases Analyses can be found here:
-
-
- Assessment instrument for teaching in random case analysis
-
Aims
The majority of our training group were either new or intending trainers. The group realised that further development of training skills amongst trainers was an ideal topic to be addressed. All trainers in the group recognised the benefits of using videos to improve trainee consultations and it was felt that there was scope for using video taped teaching sessions to develop training skills.
Random case analysis (RCA)
RCA was felt to be the teaching method most suitable for video taping. Cases presented usually took a shorter time than a topic based seminar. RCA was performed routinely by all trainers in the group but it was felt that our skills could be improved. It was also felt that the unpredictability of problems raised by RCA was advantageous as this feature tests the versatility of the trainer.
Teaching skills
The group initially reviewed a piece of work entitled “Principles and Conditions for Adult Learning”(l). This article discussed characteristics of learning and conditions which facilitated learning. Secondly the group spent time discussing the necessary components of a good tutorial. Thirdly, the group explored their experiences of good and bad teaching trying to define why a particular teaching session was either effective or ineffective. Ultimately four key areas were identified.
These were:
- Process
- Ambience
- Teaching skills
- Feedback
Structure and development of the instrument
Each trainer volunteered to look at one of the key areas identified above. The group had recognised that there were similarities between doctor/patient consultations and trainer/trainee teaching sessions. Members of the group had some experience in using the consultation assessment instrument developed by Dr. James Cox (2) and it was felt that this format could be adapted to assess RCA teaching sessions. The consultation assessment instrument uses different characteristics defined by good and bad behaviour separated by six point bipolar scales. Using similar principles each trainer developed a number of stems with good and bad characteristics within their key area. These were fed back to and modified by the group. This led to a production of an instrument that was initially workable but had an excessive number of stems. Videos of RCA sessions between the respective trainers and trainees were examined using the instrument and discussed by the group. This allowed for checking the relevance to teaching of each stem. Wording of stems was gradually modified mainly to clarify interpretation of a characteristic and thereby improving the instruments reliability. A number of stems were culled either because there was overlap with other stems or the characteristic was not thought to be relevant.
The trainer group invited an educationalist and the local course organiser to meetings and their input was particularly helpful.
Once the instrument was finalised each trainer provided further video tapes allowing formative assessment of their own teaching skills by the rest of the group.
Positive benefits
The group recognised that development of the instrument had a number of benefits.
- The project gave the group a clear direction throughout the development period.
- Discussions had helped our understanding of the necessary skills involved in teaching and assessment.
- All trainers felt that their teaching skills had improved particularly in feedback and summarising (Dl and D2).
The group recognised that the instrument would be of continued benefit to the development of training skills through formative trainer assessment.
Weaknesses
Use of the instrument showed some variability in the responses of the assessors (ie. other members of the group) which led to concerns about the reliability of the instrument. Certain stems cropped up frequently for causing this problem and the usual difficulty we had was in the exact interpretation of the stem (eg A2, Cl and C6). The stems involved were all thought to be extremely relevant to teaching and we were therefore reluctant to drop them from the instrument. Only rarely would differences in opinion extend across more than l or 2 points on each stem This variation could limit the use of such an instrument to formative assessment only.
Concern was also expressed that this method may not be sensitive enough to identify some of the more subtle non-verbal communication between a trainer and trainee.
The future
The group intends to use the instrument on a regular basis to maintain and further develop teaching skills. We hope that other trainer groups will try and use the instrument and feedback to us their experiences and suggestions for modifications.
The Group has not to-date sought a collective trainee view on the instrument outside the involvement of our personal trainees. Clearly the effectiveness of teaching is an important area and with increased use trainees views should be sought.
References
- Learning. J J Guilbert. Educational Handbook for Health Personnel. 6th Edition. World Health Organisation 1992 3.23-3.27.
- An Instrument for Assessment of Video Tapes of Genera1 Practitioners Performance. J. Cox and H. Mulholland. BMJ 17th April, 1993 Volume 306 1043-1046.