NADD Bulletin Volume III Number 6 Article 1

Complete listing

Determining the Effectiveness of Psychotherapy with Persons with Mental Retardation: The Need to Move Toward Empirically Based Research

H. Thompson Prout, Ph.D.; Kathleen M. Chard, Ph.D.; Karen M. Nowak-Drabik, M.S.; Dawn M. Johnson, M.A.

The effectiveness of psychotherapy has been a controversial issue among mental health professionals for many years. Psychotherapy involves a wide range of techniques and methods, including techniques labeled counseling, training, psychoeducational intervention, etc. However, many investigators have utilized an encompassing definition originally offered by Meltzoff and Kornreich (1970):

• The informed and planful application of tech	niques derived from established psychological principles, by a person qualified through training and experience to understand these techniques with the intent of assisting individuals to modify such person characteristics as feelings, values, attiudes, and behaviors which are judged by the therapist to maladaptive or maladjustive. (p. 6)

This definition involves direct, face-to-face interaction between client and therapist and does not include interventions that are indirectly delivered by the therapist or professional. Interventions such as consultation, behavior modification, and environmental manipulation are not included in the definition. Further, interventions that are not delivered primarily by mental health professionals are also not included. This would exclude interventions that are conducted primarily by direct care human service workers and teachers.

Historically, Eysenck (1952, 1965) and Rachman (1971) reviewed the effectiveness of psychotherapy with adults, while Levitt (1957, 1963, 1971) addressed the issue with children. These reviews essentially concluded that treatment with psychotherapy yielded no or minimal benefits when compared to untreated individuals. This view has changed since the Smith and Glass (1977, 1980) meta-analysis of the effectiveness of psychotherapy. This analysis pointed to the general effectiveness of psychotherapy. The Smith and Glass analysis has been followed by numerous additional analyses that have continued to point to effectiveness. Unfortunately, to date, there have been relatively few studies of effectiveness of psychotherapy with persons with mental retardation. The purpose of this article is to delineate some of the issues relative to establishing a firmer base of research in therapeutic interventions with persons with mental retardation and to provide some suggestions for improving research in this area.

Status of the Research on Psychotherapy with Persons with Mental Retardation

Reviews of research on the efficacy of psychotherapy with persons with mental retardation have consistently yielded the conclusion that psychotherapy is not effective, or at best, the question remains unresolved (Butz, Bowling, & Bliss, 2000; Matson, 1984; Nuffield, 1983; Prout & Strohmer, 1994; Sternlicht, 1965). Additionally, these reviews have also found that the research literature is marginal from an empirical rigor standpoint with the studies tending to be more qualitative and descriptive in nature. This lack of rigor, however, has been an area of concern in other realms of psychotherapy research. Many general books and resources on counseling and psychotherapy do not discuss issues related to persons with developmental disabilities (Prout & Strohmer, 1994). Matson (1984) noted that mental retardation is routinely used as an exclusion criteria to eliminate potential subjects from general psychotherapy effectiveness studies.

Prout and Strohmer (1994) noted an interesting interpretation of the earlier psychotherapy literature. The earlier conclusions about the general ineffectiveness of psychotherapy (Eysenck, 1952, 1965; Levitt, 1957, 1963, 1971; Rachman, 1971) seemed to center the conclusions on the failure of the therapeutic techniques. In contrast, similar findings of ineffectiveness in the same time frame tended to conclude the techniques were ineffective due the characteristics and limitations of the persons with mental retardation. Prout and Strohmer labeled this a “kill the messenger” error in logic. The attribution of the ineffectiveness to mental retardation was unjustified. In part due to this incorrect logical reasoning, Prout and Strohmer concluded that the issue of general effectiveness of psychotherapy with persons with mental retardation remained unanswered.

Butz, Bowling and Bliss (2000) noted that despite the fact that many persons with mental retardation are living in the community, there continues to be “a lack of scientific inquiry into the utility of outpatient psychotherapy.” (p. 43) They offer several reasons for this lack of research;

• Diagnostic overshadowing and the continued perception among mental health professional that intellectual deficits account for concurrent emotional symptomatology.

• The assumption that persons with mental retardation are immune to mental illness.

• Therapist bias in viewing persons with developmental disabilities as lacking abilities to understand therapeutic concepts and/or to become overly dependent.

• Dichotomization of mental health and mental retardation funding and regulatory agencies.

Recently, Prout and Nowak-Drabik (1999, 2000) conducted a large systematic review of a wide range of studies of the effectiveness of psychotherapy with persons with mental retardation. Due to the nature of the studies, it was not possible to conduct a formal meta-analysis. Using standard meta-analysis criteria, too many of the studies would have been eliminated and the analysis could only have been done with a small number of studies. Prout and Nowak-Drabik identified approximately 90 studies that were published from 1968-1998 that in some way described results of therapeutic interventions with persons with mental retardation. The studies represented a wide range of reports from case studies to experimental/control group designs. Each study was then rated in terms of outcome and general effectiveness by a panel of experts in therapeutic interventions. The overall results indicated that psychotherapy with persons with mental retardation yields moderate change and is moderately effective.

The panel reviewers in the Prout and Nowak-Drabik (1999, 2000) study also offered some general impressions on the quality of the overall body of literature on the effectiveness of therapeutic interventions with persons with mental retardation. Among their impressions were:

• The area is dominated by case studies and single subject designs.

• There are too few traditional controlled comparison studies or clinical trials.

• Interventions tend to be poorly or vaguely described.

• Very few studies seem to use treatment manuals or treatment protocols to guide the therapists in treatment.

• Studies that claim to be evaluating interventions from a certain theoretical orientation do not describe procedures to confirm that the intervention actually followed theory-based strategies. Similarly, few studies included treatment integrity procedures to assess adherence.

• Outcome data is often vaguely described or omitted.

• Outcome measures often are marginally related to the intervention and are often investigator-developed.

• Demographics and client characteristics are poorly and vaguely described.

• Conclusions appear to be based on “weak” data.

Despite these limitations, the panel reviewers did feel that the body of literature generally supported the use of psychotherapeutic interventions with persons with mental retardation.

Standards for Clinical Research

As noted, the issue of the effectiveness of psychotherapeutic interventions has a long and controversial history in the mental health disciplines. Recently, there has been an attempt to improve the quality and generalizability of treatment outcome studies. Chambless et al. (1998) have developed a set of “gold standards” for clinical trials, while the Clinical Psychology Division of the American Psychological Association have undertaken the task of creating and maintaining a list of empirically based psychological treatments for specific target populations. The list is guided by a set of criteria for well-established and probably efficacious treatments, i.e., treatments that are beneficial for specific clients in well-controlled outcome studies. Additionally, Chambless and Hollon (1998) and Herbert (2000) have attempted to further describe empirically supported therapies in their new article. The key points across these standards include:

• Demographics – including random subjects that represent the general population.

• Outcome measures – including reliable, valid and commonly used measures.

• Diagnosis – thoroughly describing disorders that are being treated and assessing for comorbid diagnoses.

• Manual – utilizing a structured treatment manual that can easily be followed.

• Statistics – performing analyses that are appropriate for the data obtained.

• Results – establishing clinical significance as well as statistical significance by performing “end-state-functioning” analyses.

• Adherence/Competence – have an external evaluator conduct adherence and competence checks on both assessment and therapy procedures.

Implications for Research on Counseling/Psychotherapy with Persons with Mental Retardation

Due to the nature of the incidence rates of mental retardation and considering the incidence rates of psychiatric disorder, we posit that a wide range of research approaches will continue to be used in the study of the effectiveness of interventions with persons with mental retardation. While Prout and Nowak-Drabik noted that the effectiveness literature was dominated by case studies and single subject designs, it appears that these types of studies will still have a place in the mental retardation/ psychotherapy area. Well-designed case studies with well-described procedures and outcomes do contribute to our understanding of treatment.

Research can be improved and made more meaningful by adapting the clinical research standards discussed above and making efforts to improve on the deficiencies of the existing research base. We offer the following as suggestions or guidelines for improving counseling/ psychotherapy outcome studies (case studies, single subject, group designs) with persons with mental retardation:

• Diagnoses and Demographics: Descriptions of clients need to be made more specific on several levels. In addition to general demographics (i.e., age, sex), level of mental retardation with IQ’s (if available) should be included. Additional information such as syndrome and medical conditions/problems should be noted. The diagnostic frameworks suggested by the American Psychiatric Association (DSM-IV) (1994) or the American Association on Mental Retardation provide much of this additional useful information. Documented psychiatric diagnoses should be included with specification of the associated presenting problems. If severity levels are recorded, those should be reported. The nature of the individuals’ environments (school, work, living situation) should also be specified. Other treatment considerations such as medication or involvement in an additional behavioral program are relevant.

• Outcome Measures: Studies of effectiveness of psychotherapy with persons without mental retardation appear to use a broader array of outcome measures. Studies with persons with mental retardation tend to rely heavily on behavioral and informant-based measures to assess outcome. General efficacy studies appear to use more self-report methodology. For example, meta-analyses of school-based psychotherapy and counseling (Prout & Demartino, 1986; Prout & Prout, 1998) found that self-report measures were the most frequently used outcome measures with child and adolescent populations in the schools. This is relevant since children’s functional academic levels and capacity to respond may be similar to older students and adults with mental retardation. Additionally, self-report measures may address counseling-related content more sensitively and directly than behavioral and/or informant measures. Available self-report instrumentation include the Self-Report Inventory of the Emotional Problems Scales (Prout & Strohmer, 1991) which assesses anxiety, depression, low self-esteem, impulse control, and thought/behavior disorder, and the self-report administration of the Psychopathology Instrument for Mentally Retarded (PIMRA) (Matson, 1988). Adaptations or adapted administration procedures of general measures may also be viable in a non-diagnostic, progress/outcome monitoring mode. For example, Prout and Schaeffer (1985), studying depression in persons with mild mental retardation, used adapted versions of several self-report measures of depression. Despite using different self-rating formats, they concluded that persons with mild retardation could reliably report on their affective states. Overall, it appears that outcome assessment could be expanded in studies, with more utilization of objective self-report measures.

• Manualized Treatment Guides: Many of the counseling and psychotherapy interventions that have been applied with persons with mental retardation are poorly described and detailed in journal articles. In many cases, the only description is that the clients received some type of counseling or skill training. The “gold standards” described above have placed increased emphasis on development of clear and well-described treatments, often in the form of a treatment manual/guide. In fact, to qualify as an “empirically supported, ” a manual or other detailed guide is required. The availability of a manual makes replication possible and makes the techniques and intervention strategies more readily available to practitioners. Manuals describe typical number of sessions, length and format, number of participants if group, specific strategies and techniques. These treatments are often more highly structured and may include specific activities and/or script-like guides. An issue related to manualized treatment guides is the issue of treatment adherence and competence. Adherence refers to documentation or validation that the treatment manual or guidelines were actually followed, i.e. treatment integrity. Competence addresses the skill level of the professional delivering the intervention. In other words, if a study indicates that it is assessing the effectiveness of reality therapy, it needs to be established that the therapists consistently applied the tenets of reality therapy. Additionally, there needs to be verification that the treatment manual was followed and the services delivered represent the strategies described. Adherence may be assessed by a review of manuals and guideline by persons with expertise in the area but who are external/independent to the study. Adherence is also assessed by reviews (often taped) of actual implementation of interventions; competence can be similarly addressed.

• Statistics and Results: As stated previously, we advocate the utilization of a wide range of methodology and designs to address the effectiveness of therapeutic interventions with persons with mental retardation. With case study and single subject methodology, objective data needs to be presented. While this may only yield an “eyeball” analysis, it preferable to the more global, vague, and general statements of improvement. Depending on the design, there are a variety of statistical options varying with the sophistication and size of the study. End of treatment outcome/status, with an emphasis on analysis of meaningful and clinically relevant variables is indicated. Where possible, follow-up analysis is also suggested. As noted above, we feel that more utilization of self-report measures is indicated. Typically, analysis of multiple outcome measures will yield the most meaningful results.

Summary

Professionals who conduct counseling and psychotherapy interventions with persons with mental retardation obviously see some value and client benefit in these services. Recent evidence (Prout & Nowak-Drabik, 1999, 2000) points to some degree of effectiveness and benefit. However, the nature of the available research is meager, particularly when compared to contemporary standards of clinical outcome research. We advocate that researchers and practitioners conducting interventions attempt to bring their studies and interventions more in line with these contemporary standards. This would yield better interventions, more appropriate matching of client problem with therapeutic technique, broader application of techniques, and more acceptance of counseling and psychotherapy as important components of overall treatment plans.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Butz, M. R., Bowling, J. B., & Bliss, C. A. (2000). Psychotherapy with the mentally retarded: A review of the literature and its implication. Professional Psychology: Research and Practice, 31, 42-47.

Chambless, D. L. & Hollon, S. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7-18.

Chambless, D. L, Baker, M. J., Baucom, D. H., Beutler, L. E., Calhoun, K. S., Crits-Christoph, P., Daiuto, A., DeRubeis, R., Detweiler, J., Haaga, D., Bennett, S., McCurry, S., Mueser, K. T., Pope, K., Sanderson, W. C., Shoham, V., Stickle, T., Williams, D. A., & Woody, S. R. (1998). Update on empirically validated therapies, II. The Clinical Psychologist, 51, 3-16.

Eysenck, H. J. (1952). The effects of psychotherapy: An evalution. Journal of Consulting Psychology, 16, 319-324.

 Eysenck, H. J. (1965). The effects of psychotherapy. Journal of Psychology, 1, 97-118.

Herbert, J. D. (2000). Defining empirically supported treatments: Pitfalls and possible solutions. The Behavior Therapist, 23, 113-122.

Levitt, E. E. (1957). The results of psychotherapy with children: An evaluation. Journal of Consulting Psychology, 21, 186-189.

Levitt, E. E. (1963). Psychotherapy with children: A further evaluation. Behavior Research and Therapy, 60, 326-329.

Levitt, E. E. (1971). Research of psychotherapy with children. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (pp. 474-493). New York: Wiley.

Matson, J. L. (1984). Psychotherapy with persons who are mentally retarded. Mental Retardation, 22, 170-175.

Matson, J. L. (1988). The Psychopathology Instrument for Mentally Retarded Adults (PIMRA). Columbus, OH: International Diagnostic Systems.

Nuffield, E. J. (1983). Psychotherapy. In J. L. Matson & J. A. Mulick (Eds.), Handbook of Mental Retardation (pp. 351-368). New York: Pergamon Press.

Meltzoff, J., & Kornreich, M. (1970). Research in Psychotherapy. New York: Atherton.

Prout, H. T., & DeMartino, R. (1986). A meta-analysis of outcome studies of school-based counseling and psychotherapy. Journal of School Psychology, 24, 285-292.

Prout, H. T. & Nowak-Drabik, K. M. (1999, Nov.). The effectiveness of psychotherapy with persons with mental retardation: Status of the research. Presentation the 16th Annual Conference of the National Association for the Dually Diagnosed, Niagara Falls. Ontario, Canada.

Prout, H. T. & Nowak-Drabik, K. M. (2000). An evaluation of the effectiveness of psychotherapy with persons with mental retardation, Manuscript in Submission.

Prout, S. M., & Prout, H. T. (1998). A meta-analysis of school-based studies of counseling and psychotherapy: An update. Journal of School Psychology, 36, 121-136.

Prout, H. T., & Schaeffer, B. M. (1985). Self-reports of depression among mentally retarded community-based adults. American Journal of Mental Deficiency, 90, 220-222.

For further information:

H. Thompson Prout, Ph.D.
247 Dickey Hall
College of Education
University of Kentucky
Lexington, KY 40508-0017
e-mail: htprout@pop.uky.edu