An Experimental Study of Procedures to Enhance Ratings of Fidelity to an Evidence-Based Family Intervention

详细信息    查看全文

• 作者：Justin D. Smith ; Thomas J. Dishion ; Kimbree Brown ; Karina Ramos…
• 关键词：Competence ; Family check ; up ; Fidelity ; Implementation ; Observational coding
• 刊名：Prevention Science
• 出版年：2016
• 出版时间：January 2016
• 年：2016
• 卷：17
• 期：1
• 页码：62-70
• 全文大小：334 KB
• 参考文献：Achenbach, T. M. (1991). Manual for the child behavior checklist/4-18 and 1991 profile. Burlington: University of Vermont, Department of Psychiatry.
  Berkel, C., Mauricio, A. M., Schoenfelder, E. N., & Sandler, I. N. (2011). Putting the pieces together: An integrated model of program implementation. Prevention Science, 12, 23–33. doi:10.​1007/​s11121-010-0186-1 .PubMed CrossRef
  Brody, G. H., Murry, V. M., Gerrard, M., Gibbons, F. X., Molgaard, V., McNair, L., & Neubaum-Carlan, E. (2004). The strong African american families program: translating research into prevention programming. Child Development, 75, 900–917. doi:10.​2307/​3696600 .PubMed CrossRef
  Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2, 40. doi:10.​1186/​1748-5908-2-40 .PubMed PubMedCentral CrossRef
  Chiapa, A., Smith, J. D., Kim, H., Dishion, T. J., Shaw, D. S., & Wilson, M. N. (2015). The trajectory of fidelity in a multiyear trial of the family check-up predicts change in child problem behavior. Journal of Consulting and Clinical Psychology. doi:10.​1037/​ccp0000034 .PubMed PubMedCentral
  Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. doi:10.​1037/​1040-3590.​6.​4.​284 .CrossRef
  Cross, W. F., & West, J. C. (2011). Examining implementer fidelity: Conceptualizing and measuring adherence and competence. Journal of Children’s Services, 6, 18–33. doi:10.​5042/​jcs.​2011.​0123 .PubMed PubMedCentral CrossRef
  Dishion, T. J., & Stormshak, E. A. (2007). Intervening in children's lives: An ecological, family-centered approach to mental health care. Washington, DC: American Psychological Association.CrossRef
  Dishion, T. J., Shaw, D. S., Connell, A., Gardner, F. E. M., Weaver, C., & Wilson, M. (2008). The family check-up with high-risk indigent families: Preventing problem behavior by increasing parents’ positive behavior support in early childhood. Child Development, 79, 1395–1414. doi:10.​1111/​j.​1467-8624.​2008.​01195.​x .PubMed PubMedCentral CrossRef
  Dishion, T.J., Knutson, N., Brauer, L., Gill, A., & Risso, J. (2010). Family Check-Up: COACH ratings manual. Unpublished coding manual. Available from the Child and Family Center, 6217. Eugene: University of Oregon OR 97403.
  Forgatch, M. S., & DeGarmo, D. S. (2011). Sustaining fidelity following the nationwide PMTO™ implementation in Norway. Prevention Science, 12, 235–246. doi:10.​1007/​s11121-011-0225-6 .PubMed PubMedCentral CrossRef
  Forgatch, M. S., DeGarmo, D. S., & Beldavs, Z. G. (2005a). An efficacious theory-based intervention for stepfamilies. Behavior Therapy, 36, 357–365. doi:10.​1016/​S0005-7894(05)80117-0 .PubMed PubMedCentral CrossRef
  Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005b). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behavior Therapy, 36, 3–13. doi:10.​1016/​S0005-7894(05)80049-8 .PubMed PubMedCentral CrossRef
  Henggeler, S. W., Schoenwald, S. K., Liao, J. G., Letourneau, E. J., & Edwards, D. L. (2002). Transporting efficacious treatments to field settings: The link between supervisory practices and therapist fidelity in MST programs. Journal of Clinical Child & Adolescent Psychology, 31, 155–167. doi:10.​1207/​S15374424JCCP310​2_​02 .CrossRef
  Hogue, A., Liddle, H. A., Becker, D., & Johnson-Leckrone, J. (2002). Family-based prevention counseling for high-risk young adolescents: Immediate outcomes. Journal of Community Psychology, 30, 1–22. doi:10.​1002/​jcop.​1047 .CrossRef
  Hogue, A., Liddle, H. A., Singer, A., & Leckrone, J. (2005). Intervention fidelity in family‐based prevention counseling for adolescent problem behaviors. Journal of Community Psychology, 33, 191–211. doi:10.​1002/​jcop.​20031 .CrossRef
  Hogue, A., Henderson, C. E., Dauber, S., Barajas, P. C., Fried, A., & Liddle, H. A. (2008). Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76, 544–555. doi:10.​1037/​0022-006x.​76.​4.​544 .PubMed PubMedCentral CrossRef
  Huey, S. J., Jr., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G. (2000). Mechanisms of change in multisystemic therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology, 68, 451–467. doi:10.​1037/​0022-006x.​68.​3.​451 .PubMed CrossRef
  Hukkelberg, S. S., & Ogden, T. (2013). Working alliance and treatment fidelity as predictors of externalizing problem behaviors in parent management training. Journal of Consulting and Clinical Psychology, 81, 1010–1020. doi:10.​1037/​a0033825 .PubMed CrossRef
  Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
  Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn & Bacon.
  Knutson, N. M., Forgatch, M. S., & Rains, L. A. (2003). Fidelity of implementation rating system (FIMP): The training manual for PMTO. Eugene: Oregon Social Learning Center.
  Landsverk, J. A., Brown, C. H., Chamberlain, P., Palinkas, L. A., Ogihara, M., Czaja, S., & Horwitz, S. M. (2012). Design and analysis in dissemination and implementation research. In R. C. Brownson, G. A. Colditz, & E. K. Proctor (Eds.), Dissemination and implementation research in health: Translating research to practice (pp. 225–260). New York: Oxford University Press.CrossRef
  Lee, C.-Y., August, G., Realmuto, G., Horowitz, J., Bloomquist, M., & Klimes-Dougan, B. (2008). Fidelity at a distance: Assessing implementation fidelity of the early risers prevention program in a going-to-scale intervention trial. Prevention Science, 9, 215–229. doi:10.​1007/​s11121-008-0097-6 .PubMed CrossRef
  Meier, S. T. (1999). Training the practitioner-scientist: Bridging case conceptualization, assessment, and intervention. The Counseling Psychologist, 27, 846–869. doi:10.​1177/​0011000099276008​ .CrossRef
  Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people for change (2nd ed.). New York: Guilford Press.
  Muse, K., & McManus, F. (2013). A systematic review of methods for assessing competence in cognitive–behavioural therapy. Clinical Psychology Review, 33, 484–499. doi:10.​1016/​j.​cpr.​2013.​01.​010 .PubMed CrossRef
  Perepletchikova, F., & Kazdin, A. E. (2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12, 365–383. doi:10.​1093/​Clipsy/​Bpi045 .
  Prado, G., Pantin, H., Schwartz, S. J., Lupei, N. S., & Szapocznik, J. (2006). Predictors of engagement and retention into a parent-centered, ecodevelopmental HIV preventive intervention for Hispanic adolescents and their families. Journal of Pediatric Psychology, 31, 874–890. doi:10.​1093/​jpepsy/​jsj046 .PubMed CrossRef
  R Core Team (2012). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Retrieved from http://​www.​R-project.​org/​ .
  Rao, C.R. (2009). Linear statistical inference and its applications (Vol. 22): Wiley.
  Safran, J. D., Muran, J. C., Samstag, L. W., & Stevens, C. (2001). Repairing alliance ruptures. Psychotherapy: Theory, Research, Practice, Training, 38, 406–412. doi:10.​1037/​0033-3204.​38.​4.​406 .CrossRef
  Schoenwald, S. K., & Henggeler, S. W. (2004). A public health perspective on the transport of evidence-based practices. Clinical Psychology: Science and Practice, 11, 360–363. doi:10.​1093/​clipsy.​bph092 .
  Schoenwald, S. K., Garland, A. F., Chapman, J., Frazier, S., Sheidow, A., & Southam-Gerow, M. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38, 32–43. doi:10.​1007/​s10488-010-0321-0 .PubMed PubMedCentral CrossRef
  Shrout, P., & Fleiss, J. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. doi:10.​1037/​0033-2909.​86.​2.​420 .PubMed CrossRef
  Silberschatz, G., Fretter, P. B., & Curtis, J. T. (1986). How do interpretations influence the process of psychotherapy? Journal of Consulting and Clinical Psychology, 54, 646–652. doi:10.​1037/​0022-006X.​54.​5.​646 .PubMed CrossRef
  Smith, J. D., Dishion, T. J., Shaw, D. S., & Wilson, M. N. (2013). Indirect effects of fidelity to the family check-up on changes in parenting and early childhood problem behaviors. Journal of Consulting and Clinical Psychology, 81, 962–974. doi:10.​1037/​a0033950 .PubMed CrossRef
  Smith, J. D., Stormshak, E. A., & Kavanagh, K. (2014). Results of a pragmatic effectiveness-implementation hybrid trial of the family check-up in community mental health agencies. Administration and Policy in Mental Health and Mental Health Services Research, 42, 265–278. doi:10.​1007/​s10488-014-0566-0 .CrossRef
  SPSS Statistics Version 22.0.0 (2014). Chicago: SPSS Inc.
  Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200–211. doi:10.​1037/​a0018912 .PubMed PubMedCentral CrossRef
  
• 作者单位：Justin D. Smith (1) (10)
  Thomas J. Dishion (2) (3)
  Kimbree Brown (4) (7)
  Karina Ramos (4) (8)
  Naomi B. Knoble (4) (9)
  Daniel S. Shaw (5)
  Melvin N. Wilson (6)
  
  1. Center for Prevention Implementation Methodology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
  10. Center for Prevention Implementation Methodology, Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, 750 N Lake Shore Drive, Chicago, IL, 60657, USA
  2. REACH Institute, Arizona State University, Tempe, AZ, USA
  3. Oregon Research Institute, Eugene, OR, USA
  4. Prevention Science Institute, University of Oregon, Eugene, OR, USA
  7. Oregon Social Learning Center, Eugene, OR, USA
  8. University of California at Berkeley, Berkeley, CA, USA
  9. Nemours/duPont Children’s Hospital, Wilmington, DE, USA
  5. University of Pittsburgh, Pittsburgh, PA, USA
  6. University of Virginia, Charlottesville, VA, USA
  
• 刊物主题：Public Health; Health Psychology; Child and School Psychology;
• 出版者：Springer US
• ISSN：1573-6695

文摘

The valid and reliable assessment of fidelity is critical at all stages of intervention research and is particularly germane to interpreting the results of efficacy and implementation trials. Ratings of protocol adherence typically are reliable, but ratings of therapist competence are plagued by low reliability. Because family context and case conceptualization guide the therapist’s delivery of interventions, the reliability of fidelity ratings might be improved if the coder is privy to client context in the form of an ecological assessment. We conducted a randomized experiment to test this hypothesis. A subsample of 46 families with 5-year-old children from a multisite randomized trial who participated in the feedback session of the Family Check-Up (FCU) intervention were selected. We randomly assigned FCU feedback sessions to be rated for fidelity to the protocol using the COACH rating system either after the coder reviewed the results of a recent ecological assessment or had not. Inter-rater reliability estimates of fidelity ratings were meaningfully higher for the assessment information condition compared to the no-information condition. Importantly, the reliability of the COACH mean score was found to be statistically significantly higher in the information condition. These findings suggest that the reliability of observational ratings of fidelity, particularly when the competence or quality of delivery is considered, could be improved by providing assessment data to the coders. Our findings might be most applicable to assessment-driven interventions, where assessment data explicitly guides therapist’s selection of intervention strategies tailored to the family’s context and needs, but they could also apply to other intervention programs and observational coding of context-dependent therapy processes, such as the working alliance.