No magic bullets: a systematic review of 102 trials of interventions to improve professional practice

Andrew D. Oxman, MD, MSc; Mary Ann Thomson, BHSc(PT); David A. Davis, MD; R. Brian Haynes, MD, PhD

Canadian Medical Association Journal 1995; 153: 1423-1431


This article is the fourth in a series of six that appeared in the October, November and December issues of CMAJ.

Dr. Oxman (principal investigator) is a member of the Health Services Research Unit, National Institute of Public Health, Oslo, Norway; Ms. Thomson is with the School of Occupational and Physical Therapy, Faculty of Health Sciences, McMaster University, Hamilton, Ont.; Dr. Davis is in the Office of Continuing Education and the Department of Family and Community Medicine, Faculty of Medicine, University of Toronto, Toronto, Ont.; and Dr. Haynes is in the departments of Clinical Epidemiology and Biostatistics and of Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ont.

Paper reprints of the full text may be obtained from: Dr. Andrew D. Oxman, Health Services Research Unit, National Institute of Public Health, Geitmyrsveien 75, 0462 Oslo, Norway



Objective: To determine the effectiveness of different types of interventions in improving health professional performance and health outcomes.

Data sources: MEDLINE, SCISEARCH, CINAHL and the Research and Development Resource Base in CME were searched for trials of educational interventions in the health care professions published between 1970 and 1993 inclusive.

Study selection: Studies were selected if they provided objective measurements of health professional performance or health outcomes and employed random or quasi-random allocation methods in their study designs to assign individual subjects or groups. Interventions included such activities as conferences, outreach visits, the use of local opinion leaders, audit and feedback, and reminder systems.

Data extraction: Details extracted from the studies included the study design; the unit of allocation (e.g., patient, provider, practice, hospital); the characteristics of the targeted health care professionals, educational interventions and patients (when appropriate); and the main outcome measure.

Data synthesis: The inclusion criteria were met by 102 trials. Areas of behaviour change included general patient management, preventive services, prescribing practices, treatment of specific conditions such as hypertension or diabetes, and diagnostic service or hospital utilization. Dissemination-only strategies, such as conferences or the mailing of unsolicited materials, demonstrated little or no changes in health professional behaviour or health outcome when used alone. More complex interventions, such as the use of outreach visits or local opinion leaders, ranged from ineffective to highly effective but were most often moderately effective (resulting in reductions of 20% to 50% in the incidence of inappropriate performance).

Conclusion: There are no "magic bullets" for improving the quality of health care, but there are a wide range of interventions available that, if used appropriately, could lead to important improvements in professional practice and patient outcomes.


Objectif : Déterminer l'efficacité de différents types d'intervention destinés à améliorer le rendement des professionnels de la santé et à accroître les résultats sur la santé.

Source de données : On a cherché dans MEDLINE, SCISEARCH, CINAHL et la Base de ressources en recherche et développement sur EMC des tentatives d'interventions d'éducation auprès des professions de la santé, publiées entre 1970 et 1993 inclusivement.

Sélection des études : On a retenu les études qui présentaient une mesure objective du rendement des professionnels de la santé ou des résultats sur la santé et dont le plan d'étude recourait à des méthodes aléatoires ou quasi aléatoires de répartition des sujets individuels ou des groupes. Les interventions comprenaient diverses activités comme des conférences, des visites de liaison, le recours à des personnalités locales, des vérifications, la rétroaction et les systèmes de rappel.

Extraction des données : Parmi les détails extraits des études, mentionnons le plan d'étude, l'unité d'attribution (comme le patient, le fournisseur, la pratique ou l'hôpital), les caractéristiques des professionnels de la santé visés, des interventions d'éducation et des patients (le cas échéant), et la principale mesure de résultats.

Synthèse des données : Les critères d'inclusion sont ressortis dans 102 études. Parmi les secteurs où s'est produit un changement de comportement, notons les relations générales avec les patients, les services de prévention, les pratiques de prescription, le traitement de certaines maladies comme l'hypertension ou le diabète, et le service diagnostic ou le recours à l'hôpital. Utilisées comme critères uniques, les stratégies axées uniquement sur la diffusion, comme les conférences ou l'envoi par la poste de documents non sollicités, n'ont suscité que très peu de changements en ce qui touche le comportement des professionnels de la santé ou les résultats sur la santé. Les interventions plus complexes, comme des visites de liaison ou le recours à des personnalités locales, ont donné des résultats allant de pauvres à excellents, mais, dans la plupart des cas, elles n'ont eu qu'un effet modéré (diminution de 20 % à 50 % de l'incidence d'une performance non appropriée).

Conclusion : Il n'y a aucune «formule magique» pour accroître la qualité des soins de santé, cependant il existe un vaste éventail d'interventions disponibles qui, si on les utilise comme il faut, pourraient donner lieu à d'importantes améliorations de la pratique des professionnels et des résultats au niveau des patients.

[Table of Contents]

Although the importance of research evidence in guiding clinical decision making is generally unquestioned intellectually, at a practical level patterns of medical practice often diverge from evidence-based recommendations, robbing patients of the benefits of medical research.(1-14) For example, Williamson and associates(14) found that one fifth to one half of primary care practitioners in the United States were not aware of or were not using new evidence about six common procedures. Similarly, in a systematic review of randomized controlled trials Lau and collaborators(15) found overwhelming evidence for the use of thrombolytic therapy in patients who had suffered myocardial infarction; however, these findings were often not reflected in the recommendations by experts in textbooks and review articles(16) or in practice.(17) These and other studies reflect shortcomings in the management of health science information by physicians and other health care professionals as well as other constraints on the use of research evidence.(18-20)

In 1992 Davis and colleagues(21) systematically reviewed 50 randomized controlled trials of the effectiveness of continuing medical education (CME), which they broadly defined as all the ways by which physician learning and clinical practice might be altered by educational or persuasive means. The objective of our study was to update and enhance that body of knowledge by (a) including nonphysician health care professionals, (b) applying explicit criteria for assessing study quality, (c) defining specific types of interventions, described later, (d) analysing other published systematic reviews of interventions targeted at a specific problem (e.g., vaccination delivery(22)) or provider (e.g., nurses(23)), or of a specific type of intervention (e.g., audit and feedback(24)), and (e) applying theoretic frameworks such as total quality management,(25) the PRECEDE (predisposing, reinforcing and enabling constructs in educational diagnosis and evaluation) health behaviour model26 and various social science and adult educational models27 to clarify the findings of trials. We present preliminary data resulting from these processes.

[Table of Contents]


Literature search

We added to the search strategies described by Davis and coworkers(21) by searching the following computerized bibliographic databases for the period January 1990 to December 1993: MEDLINE, SCISEARCH, CINAHL and the Research and Development Resource Base in Continuing Medical Education.(28)

We included trials that met the following criteria: the studies involved health care providers (excluding students); outcomes were measured by objective assessments of provider performance in a health care setting or by any health outcome; and the study designs were such that individual subjects or groups were assigned to one of two or more alternative interventions or to a control group by means of random allocation or some quasi-random method of allocation (e.g., alternation). We included the following types of intervention:

The inclusion criteria were subject to a pilot study by all of us on a sample of five studies. Two of us (A.D.O. and M.A.T.) then independently applied the criteria to all potentially relevant studies for all 10 types of intervention. Disagreements were resolved by consensus.

Assessment of trial quality

The following criteria were used to assess the quality of the trials: concealment of random allocation (protection against selection bias); completeness of follow-up; blind outcome assessment; reliable outcome, indicated by two or more raters with at least 90% agreement or achievement of a kappa value greater than 0.8; baseline measurement of performance or health outcomes measured before intervention; and assurance that contamination was unlikely (i.e., that the control group did not receive the experimental interventions). All four of us tested these on a sample of 10 studies, and then one of us (M.A.T.) applied them to each study included in the review.

Data collection

The details extracted from the studies included, but were not limited to, the following: whether the study was a randomized controlled trial (parallel groups), a quasi-randomized controlled trial such as one using alternative allocation, or a crossover study; the unit of allocation (e.g., patient, provider, practice, hospital); the characteristics of the participating providers and patients; the characteristics of the intervention, including the content, format, source, recipient, setting and timing; and a description of the main outcome measure.

We attempted to obtain sufficient information to calculate the mean proportion (per provider) of times that performance was appropriate for each patient or episode of care in each study group. We also tried to obtain the proportion of baseline performance that was adequate and the absolute and relative percent change in performance for each study group.

[Table of Contents]


We identified 102 trials(29-130) investigating one or more interventions targeted at improving the performance of health care professionals. The number of trials assessing each type of intervention is shown in Table 1. The comparison groups by type of intervention are shown in Table 2. Table 3 lists the main type of behaviour targeted in each study.

Effectiveness of interventions

[Table of Contents]


Interventions to improve professional performance are complex, and any cogent interpretation of the results of these trials requires a disentangling of the variation in the characteristics of the targeted professionals, the interventions studied, the targeted behaviours and the study designs. Because of these variations and inadequacies in reporting, it is not possible to compare or make definitive conclusions about the effects of specific types of intervention, or to measure the size of the effects. Nevertheless, several interventions have been found to improve provider performance and, to a lesser degree, health outcomes. At the same time, some common forms of continuing education have either not been tested (e.g., journal reading) or have been tested and found wanting (e.g., unsolicited mailings).

It is helpful to draw an analogy between trials of interventions to improve the performance of health care professionals and drug trials. There are (arguably) no wonder drugs; often several medications are needed, along with lifestyle or environmental changes, to effect clinically important changes in health status. It is the same with the alteration of health professional performance: many interventions have modest or negligible practical effects when used alone. However, when coupled with other strategies the effects may be cumulative and significant.

Furthermore, just as there is a need for rational drug prescribing, there is a need in the area of health professional performance to include appropriate diagnostic strategies (to determine the reasons for suboptimal performance and to identify barriers to change) and to select carefully the interventions most likely to be effective in light of the diagnosed problem. For example, reminders are likely to be effective only if not having the right information at the right time is an important cause of suboptimal performance. These diagnostic strategies bear a direct relation to quality-assurance activities. Closer collaboration of researchers in the areas of health professional performance, health services and quality assurance appears to be both desirable and necessary.(21) Future efforts should aim to improve and standardize reporting of trials of interventions to improve professional performance. A broad framework should be developed for designing and selecting appropriate interventions across the wide range of professional activities in which gaps between evidence and practice are found. An important initiative in this direction is the formation of the Cochrane Collaboration on Effective Professional Practice, which aims to prepare and keep up-to-date systematic reviews of the effects of interventions to improve professional practice.(131)

There are no "magic bullets" for improving the quality of health care. There are, however, a wide range of interventions available that, if used appropriately, could lead to substantial improvements in clinical care derived from the best available evidence.

We thank the following for their financial support: the North Thames Regional Health Authority Research and Development Programme, National Health Service, London, England; the UK Cochrane Centre, Oxford, England; and Health Canada, Ottawa.

[Table of Contents]


  1. Sherrington AM: An annotated bibliography of studies on the flow of medical information to practitioners. Methods Inf Med 1966; 2 (5): 45-57
  2. Chalmers TC: The impact of controlled trials on the practice of medicine. Mount Sinai J Med 1974; 41: 753-759
  3. Stross JK, Harlan WR: The dissemination of new medical information. JAMA 1979; 241: 2622-2624
  4. Goodwin JS, Foodwin JM: Failure to recognize efficacious treatments: a history of salicylate therapy in rheumatoid arthritis. Perspect Biol Med 1981; 25: 78-92
  5. Schade RR, Donaldson RM: How physicians use cimetidine: a survey of hospitalized patients and published cases. N Engl J Med 1981; 304: 1281-1284
  6. Rockette HE, Redmond CK, Fisher B: Impact of randomized clinical trials on therapy of primary breast cancer: the NSABP overview. Control Clin Trials 1982; 3: 209-225
  7. Garnier HS, Flamant R, Fohanno C: Assessment of the role of randomized clinical trials in establishing treatment policies. Control Clin Trials 1982; 3: 227-234
  8. Fisher LD, Kennedy JW: Randomized surgical clinical trials for treatment of coronary artery disease. Control Clin Trials 1982; 3: 335-338
  9. Friedman L, Wenger NK, Knatterude GL: Impact of the coronary drug project findings on clinical practice. Control Clin Trials 1983; 4: 513-522
  10. Fineberg HV: Effects of clinical evaluation on the diffusion of medical technology. In Committee for Evaluating Medical Technologies in Clinical Use: Assessing Medical Technologies, National Academy Press, Washington, 1985: 176-210
  11. Lewis C: Disease prevention and health promotion practices of primary care physicians in the United States. Am J Prev Med 1988; 4: 9-16
  12. Bass MJ, Elford WE: Preventive practice patterns of Canadian primary care physicians. Am J Prev Med 1988; 4: 17-23
  13. Stross JK: Relationships between knowledge and experience in the use of disease-modifying antirheumatic agents: a study of primary care practitioners. JAMA 1989; 262: 2721-2723
  14. Williamson JW, German PS, Weiss R et al: Health science information management and continuing education of physicians: a survey of US primary care practitioners and their opinion leaders. Ann Intern Med 1989; 110: 151-160
  15. Lau JL, Antman EM, Jimenez-Silva J et al: Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 1992; 327: 248-254
  16. Antman EM, Lau J, Kupelnick B et al: A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. JAMA 1992; 268: 240-248
  17. Ketley D, Woods KL: Impact of clinical trials on clinical practice: example of thrombolysis for acute myocardial infarction. Lancet 1993; 342: 891-894
  18. Lomas J, Haynes RB: A taxonomy and critical review of tested strategies for the application of clinical practice recommendations: from official to individual clinical policy. Am J Prev Med 1988; 4 (4 suppl): S77-S94
  19. Eisenberg JM: Doctors' Decisions and the Cost of Medical Care, Health Administration Press Perspective, Ann Arbor, Mich, 1986
  20. Eddy DM: Clinical policies and the quality of clinical practice. N Engl J Med 1982; 307: 343-347
  21. Davis DA, Thomson MA, Oxman AD et al: Evidence for the effectiveness of CME: a review of 50 randomized controlled trials. JAMA 1992; 268: 1111-1117
  22. Gyorkos TW, Tannenbaum TN, Abrahamowicz A et al: Evaluation of the effectiveness of immunization delivery methods. Can J Public Health 1994; 85 (suppl 1): S37-S40
  23. Waddell DL: The effects of continuing education on nursing practice: a meta-analysis. J Contin Educ Nurs 1991; 22: 113-118
  24. Mugford M, Banfield P, O'Hanlon M: Effects of feedback of information on clinical practice: a review. [review] BMJ 1991; 303: 398-402
  25. Murrey KO, Gottlieb LK, Schoenbaum SC: Implementing clinical guidelines: a quality management approach to reminder systems. [review] QRB Qual Rev Bull 1992; 18: 423-433
  26. Green LW, Eriksen MP, Schor EL: Preventive practices by physicians: behavioral determinants and potential interventions. [review] Am J Prev Med 1988; 4 (4 suppl): S101-S107
  27. Lomas J: Teaching Old (and Not So Old) Docs New Tricks: Effective Ways to Implement Research Findings. No 93-4 of CHEPA Working Paper Series, McMaster University, Hamilton, Ont, 1993
  28. Davis DA, Fox RD (eds): The Physician as Learner: Linking Research to Practice, American Medical Association Press, Chicago, 1994: xiii
  29. Avorn J, Soumerai SB: Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based detailing. N Engl J Med 1983; 308: 1457-1463
  30. Evans CE, Haynes RB, Birkett NJ et al: Does a mailed continuing education program improve physician performance? Results of a randomized trial in antihypertensive care. JAMA 1986; 255: 501-504
  31. Lomas J, Enkin M, Anderson GM et al: Opinion leaders versus audit and feedback to implement practice guidelines. JAMA 1991; 265: 2202-2207
  32. Marton KI, Tul V, Sox HC: Modifying test-ordering behaviour in the outpatient medical clinic. Arch Intern Med 1985; 145: 816-821
  33. Meyer TJ, Van Kooten D, March S et al: Reduction of polypharmacy by feedback to clinicians. J Gen Intern Med 1991; 6: 133-136
  34. Maiman LA, Becker MH, Liptak GS et al: Improving pediatricians' compliance-enhancing practices: a randomized trial. Am J Dis Child 1988; 142: 773-779
  35. Wilson DM, Taylor W, Gilbert JR et al: A randomized trial of a family physician intervention for smoking cessation. JAMA 1988; 260: 1570-1574
  36. McConnell TS, Cushing AH, Bankhurst AD et al: Physician behaviour modification using claims data: tetracycline for upper respiratory tract infection. West J Med 1982; 137: 448-450
  37. Avorn J, Soumerai SB, Everitt DE et al: A randomized trial of a program to reduce the use of psychoactive drugs in nursing homes. N Engl J Med 1992; 327: 168-173
  38. Soumerai SB, Salem-Schatz S, Avorn J et al: A controlled trial of educational outreach to improve blood transfusion practice. JAMA 1993; 270: 961-966
  39. Cummings SR, Coates TJ, Richard RJ et al: Training physicians in counselling about smoking cessation, a randomized trial of the "quit for life" program. Ann Intern Med 1989; 110: 640-647
  40. Dietrich AJ, O'Connor GT, Keller A et al: Cancer: improving early detection and prevention; a community practice randomised trial. BMJ 1992; 304: 687-691
  41. Putnam RW, Curry L: Impact of patient care appraisal on physician behaviour in the office setting. CMAJ 1985; 132: 1025-1029
  42. Stross JK, Hiss RG, Watts CM et al: Continuing education in pulmonary disease for primary care physicians. Am Rev Respir Dis 1983; 127: 739-746
  43. Stross JK, Bole GG: Evaluation of an educational program for primary care practitioners on the management of osteoarthritis. Arthritis Rheum 1985; 28: 108-111
  44. Stross JK, Banwell BF, Wolf FM et al: Evaluation of an education program on the management of rheumatic diseases for physical therapists. J Rheumatol 1986; 13: 374-378
  45. Gehlbach SH, Wilkinson WE, Hammond WE et al: Improving drug prescribing in a primary care practice. Med Care 1984; 22: 193-201
  46. Hershey CO, Porter DK, Breslau D et al: Influence of simple computerized feedback on prescription charges in an ambulatory clinic. Med Care 1986; 24: 472-481
  47. Cohen SJ, Stookey GK, Katz BP et al: Encouraging primary care physicians to help smokers quit. Ann Intern Med 1989; 110: 648-652
  48. Cummings SR, Richard RJ, Duncan CL et al: Training physicians about smoking cessation: a controlled trial in private practices. J Gen Intern Med 1989; 4: 482-489
  49. Vinicor F, Cohen SJ, Mazzuca SA et al: DIABEDS: A randomized trial of the effects of physician and/or patient education on diabetes patient outcomes. J Chronic Dis 1987; 40: 345-356
  50. Magruder-Habib K, Zung WK, Feussner JR: Improving physicians' recognition and treatment of depression in general medical care. Med Care 1990; 28: 239-250
  51. Rubenstein LV, Calkins DR, Young RT et al: Improving patient function: a randomized trial of functional disability screening. Ann Intern Med 1989; 111: 836-842
  52. Jennett PA, Laxdal OE, Hayton RC et al: The effects of continuing medical education on family doctor performance in office practice: a randomized control study. Med Educ 1988; 22: 139-145
  53. White CW, Albanese MA, Brown DD et al: The effectiveness of continuing medical education in changing the behaviour of physicians caring for patients with acute myocardial infarction. Ann Intern Med 1985; 102: 689-692
  54. Putnam RW, Curry L: Physicians' participation in establishing criteria for hypertension management in the office: Will patient outcomes be improved? CMAJ 1989; 140: 806-809
  55. Mazzuca SA, Vinicor F, Einterz RM et al: Effects of the clinical environment on physicians' response to postgraduate medical education. Am Educ Res J 1990; 27: 473-478
  56. Adams ID, Chan M, Clifford PC et al: Computer aided diagnosis of acute abdominal pain: a multicentre study. BMJ 1986; 293: 800-804
  57. Barnett GO, Winickoff RN, Morgan MM et al: A computer-based monitoring system for follow-up of elevated blood pressure. Med Care 1983; 21: 400-409
  58. Bass MJ, McWhinney IR, Donner A: Do family physicians need medical assistants to detect and manage hypertension? CMAJ 1986; 134: 1247-1255
  59. Berwick DM, Coltin KL: Feedback reduces test use in a health maintenance organization. JAMA 1986; 255: 1450-1454
  60. Billi JE, Hejna GF, Wolf FM et al: The effects of a cost-
    education program on hospital charges. J Gen Intern Med 1987; 306-311
  61. Britton ML, Lurvey PL: Impact of medication profile review on prescribing in a general medicine clinic. Am J Hosp Pharm 1991; 48: 265-270
  62. Bulpitt CJ, Beilin LJ, Coles EC et al: Randomised controlled trial of computer-held medical records in hypertensive patients. BMJ 1976; 1: 677-679
  63. Buntinx F, Knottnerus JA, Crebolder HFJM et al: Reactions of doctors to various forms of feedback designed to improve the sampling quality of cervical smears. Qual Assur Health Care 1992; 4: 161-166
  64. Buntinx F, Knottnerus JA, Crebolder HFJM et al: Does feedback improve the quality of cervical smears? A randomized controlled trial. Br J Gen Pract 1993; 43: 194-198
  65. Chambers CV, Balaban DJ, Carlson BL et al: Microcomputer-generated reminders, improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract 1989; 29: 273-280
  66. Chase CR, Vacek PM, Shinozaki T et al: Medical information management: improving the transfer of research results to presurgical evaluation. Med Care 1983; 21: 410-424
  67. Chassin MR, McCue SM: A randomized trial of medical quality assurance. JAMA 1986; 256: 1012-1016
  68. Cheney C, Ramsdell JW: Effect of medical records' checklists on implementation of periodic health measures. Am J Med 1987; 83: 129-136
  69. Cohen DI, Jones P, Littenberg B et al: Does cost information availability reduce physician test usage? Med Care 1982; 20: 286-292
  70. Cohen SJ, Weinberger M, Hui SL et al: The impact of reading on physicians' non-adherence to recommended standards of medical care. Soc Sci Med 1985; 21: 909-914
  71. Cohen SJ, Christen AG, Katz BP et al: Counselling medical and dental patients about cigarette smoking: the impact of nicotine gum and chart reminders. Am J Public Health 1987; 77: 313-316
  72. Cowan JA, Heckerling PS, Parker JB: Effect of a fact sheet reminder on performance of the periodic health examination: a randomized controlled trial. Am J Prev Med 1992; 8: 104-109
  73. Davidoff F, Goodspeed R, Clive J: Changing test ordering behaviour, a randomized controlled trial comparing probabilistic reasoning with cost-containment education. Med Care 1989; 27: 45-58
  74. Dickinson JC, Warshaw GA, Gehlbach SH et al: Improving hypertension control: impact of computer feedback and physician education. Med Care 1981; 19: 843-854
  75. Everett GD, deBlois S, Chang PF et al: Effect of cost education, cost audits, and faculty chart review on the use of laboratory services. Arch Intern Med 1983; 143: 942-944
  76. Garett LE Jr, Hammond WE, Stead WW: The effects of computerized medical records on provider efficiency and quality of care. Methods Inf Med 1986; 25: 151-157
  77. Gullion DS, Tschann JM, Adamson TE et al: Management of hypertension in private practices: a randomized controlled trial in continuing medical education. J Contin Educ Health Prof 1988; 8: 239-255
  78. Headrick LA, Speroff T, Pelecanos HI et al: Efforts to improve compliance with the national cholesterol education program guidelines. Arch Intern Med 1992; 152: 2490-2496
  79. Heale J, Davis D, Norman G et al: A randomized controlled trial assessing the impact of problem-based versus didactic teaching methods in CME. Proc Res Med Educ 1988; 27: 72-77
  80. Hershey CO, Goldberg HI, Cohen DI: The effect of computerized feedback coupled with a newsletter upon outpatient prescribing charges. Med Care 1988; 26: 88-93
  81. Johnson RE, Campbell WH, Azevedo DJ et al: Studying the impact of patient drug profiles in an HMO. Med Care 1976; 14: 799-807
  82. Johnson RE, Campbell WH, Azevedo DJ et al: Examining physicians' drug order recording behavior. Med Care 1978; 16: 408-416
  83. Kopelow ML, Schnabl GK, Hassard TH et al: Assessing practicing physicians in two settings using standardized patients. Acad Med 1992; 67 (10 suppl): S19-S21
  84. Kottke TE, Brekke ML, Solberg LI et al: A randomized trial to increase smoking intervention by physicians. JAMA 1989; 261: 2101-2106
  85. Levinson W, Roter D: The effects of two continuing medical education programs on communication skills of practicing primary care physicians. J Gen Intern Med 1993; 8: 318-324
  86. Linn BS: Continuing medical education impact on emergency room burn care. JAMA 1980; 244: 565-570
  87. Litzelman DK, Dittus RS, Miller ME et al: Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols.
    J Gen Intern Med 1993; 8: 311-317
  88. Manheim LM, Feinglass J, Hughes R et al: Training house officers to be cost conscious: effects of an educational intervention on charges and length of stay. Med Care 1990; 28: 29-42
  89. Martin AR, Wolf MA, Thibodeau LA et al: A trial of two strategies to modify the test-ordering behavior of medical residents. N Engl J Med 1980; 303: 1330-1336
  90. Mayefsky JH, Foye HR: Use of a chart audit: teaching well-child care to paediatric house officers. Med Educ 1993; 27: 170-174
  91. McAlister NH, Covvey HD, Tong C et al: Randomised controlled trial of computer assisted management of hypertension in primary care. BMJ 1986; 293: 670-674
  92. McDonald CJ: Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med 1976; 84: 162-167
  93. McDonald CJ: Protocol-based computer reminders, the quality of care and the non-perfectibility of man. N Engl J Med 1976; 295: 1351-1355
  94. McDonald CJ, Wilson GA, McCabe GP: Physician response to computer reminders. JAMA 1980; 244: 1579-1581
  95. McDonald CJ, Hui SL, Smith DM et al: Reminders to physicians from an introspective computer medical record, a two-year randomized trial. Ann Intern Med 1984; 100: 130-138
  96. McDonald CJ, Hui SL, Tierney WM: Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics. MD Comput 1992; 9: 304-312
  97. McPhee SJ, Bird JA, Jenkins CNH et al: Promoting cancer screening: a randomized controlled trial of three interventions. Arch Intern Med 1989; 149: 1866-1872
  98. McPhee SJ, Bird JA, Fordham D et al: Promoting cancer prevention activities by primary care physicians: results of a randomized controlled trial. JAMA 1991; 266: 538-544
  99. Morgan M, Studney DR, Barnett GO et al: Computerized concurrent review of prenatal care. QRB 1978; 4: 33-36
  100. Nattinger AB, Panzer RJ, Janus J: Improving the utilization of screening mammography in primary care settings. Arch Intern Med 1989; 149: 2087-2092
  101. Norton PG, Dempsey LJ: Self-audit: its effect on quality of care. J Fam Pract 1985; 21: 289-291
  102. Palmer RH, Louis TA, Hsu LN et al: A randomized controlled trial of quality assurance in sixteen ambulatory care practices. Med Care 1985; 23: 751-770
  103. Perera DR, LoGerfo JP, Shulenberger E et al: Teaching sigmoidoscopy to primary care physicians: a controlled study of continuing medical education. J Fam Pract 1983; 16: 785-788
  104. Pollack MM, Getson PR: Pediatric critical care cost containment: combined actuarial and clinical program. Crit Care Med 1991; 19: 12-20
  105. Pozen MW, D'Agostino RB, Mitchell JB et al: The usefulness of a predictive instrument to reduce inappropriate admissions to the coronary care unit. Ann Intern Med 1980; 92: 238-242
  106. Pozen MW, D'Agostino RB, Selker HP et al: A predictive instrument to improve coronary-care-unit admission practices in acute ischemic heart disease: a prospective multi-center clinical trial. N Engl J Med 1984; 310: 1273-1278
  107. Restuccia JD: The effect of concurrent feedback in reducing inappropriate hospital utilization. Med Care 1982; 20: 46-62
  108. Rodman JH, Jelliffe RW, Kolb E et al: Clinical studies with computer-assisted initial lidocaine therapy. Arch Intern Med 1984; 144: 703-709
  109. Rogers JL, Haring OM: The impact of a computerized medical record summary system on incidence and length of hospitalization. Med Care 1979; 17: 618-630
  110. Rogers JL, Haring OM, Wortman PM et al: Medical information systems: assessing impact in areas of hypertension, obesity and renal disease. Med Care 1982; 20: 63-74
  111. Rosser WW, McDowell I, Newell C: Use of reminders for preventive procedures in family medicine. CMAJ 1991; 145: 807-814
  112. Sanazaro PJ, Worth RM: Concurrent quality assurance in hospital care. N Engl J Med 1978; 298: 1171-1177
  113. Sibley JC, Sackett DL, Neufeld V et al: A randomized controlled trial of continuing medical education. N Engl J Med 1982; 306: 511-515
  114. Steele MA, Bess DT, Franse VL et al: Cost effectiveness of two interventions for reducing outpatient prescribing costs. DICP 1989; 23: 497-500
  115. Stross JK, Bole GG: Evaluation of a continuing education program in rheumatoid arthritis. Arthritis Rheum 1980; 23: 846-849
  116. Tape TG, Campbell JR: Computerized medical records and preventive health care: success depends on many factors. Am J Med 1993; 94: 619-625
  117. Tape TG, Givner N, Wigton RS et al: Process in ambulatory care: a controlled clinical trial of computerized records. In Greenes R (ed): Proceedings of the 12th Annual Symposium on Computer Applications in Medical Care, SCAMC Inc, Washington, 1988: 749-752
  118. Tierney WM, Hui SL, McDonald CJ: Delayed feedback of physician performance versus immediate reminders to perform preventive care. Med Care 1986; 24: 659-666
  119. Tierney WM, McDonald CJ, Hui SL et al: Computer predictions of abnormal test results effects on outpatient testing. JAMA 1988; 259: 1194-1198
  120. Tierney WM, Miller ME, McDonald CJ: The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990; 322: 1499-1504
  121. Turner RC, Waivers LE, O'Brien K: The effect of patient-carried reminder cards on the performance of health maintenance measures. Arch Intern Med 1990; 150: 645-647
  122. Weingarten MA, Bazel D, Shannon HS: Computerized protocol for preventive medicine: a controlled self-audit in family practice. Fam Pract 198; 6: 120-124
  123. Wexler JR, Swender PT, Tunnessen WW et al: Impact of a system of computer-aided diagnosis. Am J Dis Child 1975; 129: 203-205
  124. White RH, Hong R, Venook AP et al: Initiation of warfarin therapy. J Gen Intern Med 1987; 2: 141-148
  125. White PT, Pharoah CA, Anderson HR et al: Randomized controlled trial of small group education on the outcome of chronic asthma in general practice. J R Coll Gen Pract 1989; 39: 182-186
  126. Wigton RS, Zimmer JL, Wigton JH et al: Chart reminders in the diagnosis of anemia. JAMA 1981; 245: 1745-1747
  127. Winickoff RN, Coltin KL, Morgan MM et al: Improving physician performance through peer comparison feedback. Med Care 1984; 22: 527-534
  128. Wones RG: Failure of low-cost audits with feedback to reduce laboratory test utilization. Med Care 1987; 25: 78-82
  129. Hong SW, Ching TY, Fung JPM et al: The employment of ward opinion leaders for continuing education in the hospital. Med Teach 1990; 12: 209-217
  130. Raisch DW, Bootman JL, Larson LN et al: Improving antiulcer agent prescribing in a health maintenance organization. Am J Hosp Pharm 1990; 47: 1766-1773
  131. Freemantle N, Grilli R, Grimshaw J et al: Implementing findings of medical research: the Cochrane Collaboration on Effective Professional Practice. Q Health Care 1995; 4: 45-47

[Table of Contents]


This guideline is for reference and education only and is not intended to be a substitute for the advice of an appropriate health care professional or for independent research and judgement. The CMA relies on the source of the CPG to provide updates and to notify us if the guideline becomes outdated. The CMA assumes no responsibility or liability arising from any outdated information or from any error in or omission from the guideline or from the use of any information contained in it.
CMAJ November 15, 1995 (vol 153, no 10) / JAMC le 15 novembre 1995 (vol 153, no 10)