Open Collections

Performance evaluation of a program in pharmacy continuing education

University of British Columbia

The focus of this investigation was on the evaluation of a pharmacy continuing education program and the development and assessment of written simulations as an evaluative tool. A program designed as a regular course offering by the Faculty and Division of Continuing Education in the Health Sciences to improve the primary care consulting skills of pharmacists, was evaluated in terms of improvement in the participant's real-life performance, gain in factual knowledge and subjective reaction to the educational activity. In order to measure the quality of primary care consultation, four "in-store-assessment" problems were developed which, in the opinion of a panel of pharmacy practitioners, occur daily in community pharmacy practice. These problems involved a consumer requesting assistance for either a "cold" or a "pain" complaint. Each request was accompanied by a list of possible pharmacist responses which the panel members rated as desirable or undesirable behaviours. Four subsets of these behaviours: "data gathering", "inappropriate recommendations", "appropriate recommendations" and "drug-use-counselling" were selected as performance criteria for each problem. The effect of the educational activity on the primary care consulting behaviour of program participants (N=34) at their regular place of practice, was examined by the unobtrusive administration of the problems. Assessments were conducted both before and after the program. A "non-equivalent" control group (N=39) was assessed during the same time periods. The problems were presented in a standardized manner by specially trained observers who recorded the pharmacists' behaviours on prepared performance checklists. Performance scores on each of the behavioural subsets and a total score were calculated to represent the quality of advice given by the pharmacists. The impact of the program on pharmaceutical services was determined by calculating gain scores. There was a significant improvement in the overall performance score for the pharmacists who attended the educational program. There were significant gains in the performance of course participants for "data gathering" and "appropriate recommendations" behaviours. There were no significant changes, in any of the performance areas, for the "non-equivalent" control group. Tests for factual knowledge were developed for the content areas of "cold" and "pain". These were administered to the course participants before and after instruction on these topics. There were significant gains in the recall of the information by the pharmacists. There were no significant relationships between an individual's score on these tests and performance scores on the requests for primary care. A magnitude estimation scaling technique was used by the participants to evaluate the program on several dimensions. In their opinion this program was superior in all aspects to the average continuing education course attended in the past. Four written simulations were developed with the aid of the panel of pharmacy practitioners. Each was constructed with respect to a specific request for primary care assistance. The content of the simulations approximated the content of the four problems used to assess the pharmacists' behaviours at their place of employment. The pharmacists enrolled in the program completed the simulations before and after instruction on "cold" and "pain". The performance scores on the four simulations were compared with the performance scores on the corresponding problems used during the "in-store-assessment". All four simulations had a positive correlation with the pharmacist's real-life performance. Two of the simulations had significant correlation coefficients. Selected common behaviours were compared between the written simulations and the "in-store-assessment" problems. A consistency score was calculated representing the agreement among performance on the items. For the pre-and post-measurements the respective mean consistency scores were 66.54 and 65.71 percent. There were no significant relationships between scores on the tests of factual knowledge and performance scores on the written simulations. The results suggest that this continuing education program was a success. There were improvements in real-life performance, significant gains in factual knowledge and feelings of satisfaction with the program on the part of the program participants. As well, the findings in this study indicated that written simulations hold promise as an objective evaluation tool for continuing pharmacy education and were capable of predicting real-life behaviour.

Text

2010-02-23

For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

http://hdl.handle.net/2429/20804

Administrative, Adult and Higher Education

University of British Columbia

Graduate