Evaluation and Accountability in Clinical Training

Accountability in clinical training implies a strong relationship between the training outcomes touted by a training program and the performance of its graduates. The training program and its faculty must be able to ensure that students have the competencies necessary for entering the profession and can offer competent services. In addition, responsibility for the quality and value of training must be assumed by the profession. Pressure for accountability is becoming increasingly apparent as the public learns about fraud, waste, and abuse in publicly funded pro­ grams (Fishman & Neigher, 1982). Federally supported clinical training programs have had to defend their training practices against threats of funding loss without the hard data needed to support their practices. Funding seems to have been forthcoming mostly because of our ability to demonstrate the need for clinical, counseling, and school psychol­ ogists. Graduates seeking professional careers in such applied fields demand considerable trust from their clientele and the public-at-large when they establish themselves, offer and advertise their services, make claims on public monies, and profess to do good and no harm. Neither their clien­ tele nor the public are in the position to evaluate the services of the profes­ sion or the claims made for these. (American Psychological Association lAPA], 1982, p.

1 Evaluation and Accountability in Training for Professional Psychology: An Overview.- Validating Individual Practitioners.- Evaluation of Education and Training.- Assumptions upon Which to Base an Evaluation Program.- Some Limitations for Evaluating Education and Training.- What Do We Want to Know about Education and Training?.- An Initial Research Program.- Evaluation of Education and Training and Quality Assurance.- Need for Proximal Measures of Performance.- Research Designs for Comparison Groups.- Evaluation of Education and Training in Other Professions.- Conclusions.- References.- 2 Program Evaluation: Defining and Evaluating Quality.- Nominal and Ordinal Aspects of Quality.- Functions of External Evaluation.- Evaluation of Output.- Evaluations of Process.- Metaevaluation.- Final Comments.- References.- 3 Work Samples and Simulations in Competency Evaluation.- Psychology’s Evaluation of Itself.- History and Rationale.- Definitions and Examples.- Advantages of Performance Tests.- Application of Skills.- Interactive Skills.- Detachment of Stress from the Examiners.- Content Validity.- Predictive Validity.- Less Adverse Impact.- Disadvantages of Performance Tests.- Costs.- Administrative Complexities.- Range of Knowledge.- Reliability.- Work Samples versus Simulations.- Work Samples.- Simulations.- An Experimental Contrast in Psychiatry.- Conclusions and Discussion.- References.- 4 Current Status of Training Evaluation in Clinical Psychology.- Urgency of the Issue.- Scope of the Chapter.- Defining Evaluation Questions.- Variations in Perspectives and Purposes.- Variations in Training Context.- Design Alternatives and Research Agendas.- Constructing the Clinical Psychologist.- National Findings on Training Evaluation Practices.- Methodology.- Training Evaluation Procedures.- Evaluation Impact.- Evaluation Obstacles.- Setting Variations.- Specification and Measurement of Clinical Competence.- Methodological Considerations.- The Contents and Procedures of Competency Measures.- Making Progress in Measuring Competence.- Assessing Training Processes.- The Value of Studying Training Processes.- Approaches to Process Investigation.- Linking Processes to Outcomes.- Future Directions.- References.- 5 Training and Evaluation in Counseling Psychology.- The Importance of Evaluation.- Efforts to Evaluate Training in the Helping Professions.- A Survey of Current Evaluation Practices in Counseling Psychology.- Future Directions.- References.- 6 Graduate Education and Training in Health Psychology.- Predoctoral Education in Psychology: Generic or Specialized?.- Doctoral Education in Health Psychology.- Postdoctoral Training in Health Psychology.- Employment Opportunities for Health Psychologists.- Licensure of Health Psychologists.- References.- 7 Evaluating School Psychologists and School Psychological Services.- The Early Affiliation of Educational Psychology, School Psychology, and Clinical Psychology.- The Emergence of the Specialties of Clinical Psychology and School Psychology.- Models of School Psychology.- The Clinical Diagnostic Model.- The Prevention-Promotion Model.- The Social Learning Model.- The Organizational Development Model.- The Prescriptive Intervention Model.- The Child Advocacy Model.- Models of School Psychology Services and Evaluation.- A Three-Dimensional Paradigm for Evaluation.- The Source Dimension.- The Roles Dimension.- The Skills Dimension.- Developmental Strands for Evaluating Changes in School Psychologists over Time.- Developmental Strand for Applying Psychological Theories to Assessment.- Developmental Strand for Teaching the Relevant Psychological Theories in Parent and Teacher Education.- Developmental Strand for Acquiring Competence in Conducting Research on Educational Topics.- Designing Custom Evaluations.- Conclusion.- References.- 8 Clinical Evaluation of Medical Students in U.S. Medical Schools.- Structure of Clinical Education in U.S. Medical Schools.- Clinical Courses in the First 2 Years of Medical School.- Clinical Clerkships.- Graduate Medical Education.- Evaluation of Medical Students during Clinical Education.- Content and Method of Evaluation.- Problems with Current Evaluation Practices.- Research on Methods of Evaluating Clinical Competence.- Developing an Effective System of Evaluation.- References.- 9 Improving Evaluation in Counselor Training and Credentialing through Standardized Simulations.- Traditional Evaluation Methods.- Standardized Simulations as an Evaluation Method.- Applications of Standardized Simulations in Medicine.- Types of Standardized Simulations in Counseling.- Interview Simulations.- Case Management Simulations.- Applications of Standardized Simulations.- Evaluation in Counselor Training.- Evaluation in Counselor Credentialing.- Evaluation of Counselor Training Programs.- Expanding Applications of Standardized Simulations.- Definitions of Competence.- Measurement Characteristics of Standardized Simulations.- Defining Performance Criteria.- Conclusion.- References.- 10 Accreditation: A Link between Training and Practice.- The Nature of Accreditation: Values, Objectives, and Methods.- Accreditation of Education and Training Programs in Professional Psychology.- Standards for Practice in Professional Psychology.- Accreditation of Psychological Service Delivery Settings.- Concluding Remarks.- References.- 11 Licensure and Certification of Psychologists.- Summary of Minimal Competence by State.- Type of Governmental Regulation.- Definition of the Psychologist.- Experiential Requirements.- Examinations.- Pass Point on the EPPP.- Restrictiveness.- Relationships among the Criteria of Minimal Competence.- Summary.- The National Licensure Examination.- Proper Use of the EPPP.- Establishment of a Pass Point on the EPPP.- Equating Procedure.- Content Validation of the EPPP.- Role Delineation Study.- Empirical Verification: The Scottsdale Conference.- Summary.- Content Validation of the Item Bank.- New and Other Examination Methods.- Oral Examination.- Jurisprudence and Essay Examinations.- Written Simulation.- Implications for Graduate Faculty.- References.- 12 Certification of Psychologists by the American Board of Professional Psychology.- History.- The American Board of Professional Psychology Today: Structure and Function.- Requirements for Certification by ABPP.- Training and Supervised Experience Requirements.- Other Requirements.- Professional Work Sample.- Oral Examination.- Dimensions of Competency Covered by the Oral Examination.- Award of the Diploma.- Future Trends in Credentialing by ABPP.- Alternative Examination Procedures Based on “Assessment Centers” Model.- New Areas of Professional Practice.- Certification Procedures and Criteria Established by the ABPP: Implications for Training in Professional Psychology.- Incentives to Seek the ABPP Diploma.- “Cradle-to-Grave” Concept.- Credentialing: Turf Guarding or Opportunity for Cooperation?.- References.- 13 Criterion-Referenced, Competency-Based Training in Behavior Modification.- Overview.- Definitions.- Need for Competency-Based Training.- Guideline Development.- Identification of Competencies and Assessment Criteria.- First Survey.- Second Survey.- Comparison of First and Second Surveys.- Developing Tests for Target Competencies.- Evaluation of the Assessment and Training Procedures.- Discussion.- Current Status.- Future Direction.- Refinement of Training Techniques.- Expansion of Competency Areas.- Summary.- References.- 14 Continuing Professional Education for Clinical Psychology: A Practice-Oriented Model.- Overview of the Continuing Professional Education Development Project.- Project Goals.- The Practice Audit Model.- Participation of Clinical Psychology in the Project.- The Practice Description for Clinical Psychology.- The Role Delineation Study.- The Practice Audit for Clinical Psychology.- Client Simulation.- Other Exercises.- Practice Audit Criteria.- Results of the Practice Audit Session.- The Clinical Skills Workshop.- Workshop Evaluation.- Practice-Oriented Continuing Professional Education.- Comparison of Knowledge-Based versus Practice-Oriented Approaches.- Cost Consideration.- Prospects for Practice-Oriented Continuing Professional Education.- References.