NADD Bulletin Volume IV Number 6 Article 1

Complete listing

When Staff and Consumers Disagree About Consumer Satisfaction

James R. Dudley, Ph.D.

Introduction

Consumer satisfaction studies are an important source of information for numerous purposes in both the fields of mental health and developmental disabilities. While concerns have been expressed about the accuracy of many consumers’ responses, these concerns have been largely addressed by using multiple sources of information and various types of reliability testing (Sigelman, Budd, Spanhel, & Schoenrock, 1981). Service providers, government agencies, the courts and others are increasingly depending upon the consumers’ perspectives about their lives for numerous purposes, including quality assurance studies, other types of program evaluations, court compliance issues, and to assist practitioners (Dudley, Calhoun, Ahlgrim-Delzell, & Conroy, 1998).

This report focuses on an important topic related to consumer satisfaction—how aware staff members are of the consumers’ satisfaction with their daily circumstances. While no studies could be found on this topic, two recent studies had a somewhat similar purpose, comparing consumer ratings of their circumstances to staff ratings. In other words, they investigated staff ratings rather than what staff perceived to the consumers’ ratings to be. Brown, Cosgrove, and DeSelm (1997) compared the perceptions of 19 consumers and case managers regarding the barriers interfering with the consumers’ life satisfaction. They found that case managers identified more person-oriented barriers and consumers identified more environmental barriers. A similar study by Lunsky and Benson (1997) involved interviews with 50 people, with a mild retardation label, and their staff to determine their respective ratings of social supports. Consumer and staff ratings were found to significantly correlate, particularly related to family support. Less agreement was found in their ratings of support from friends and staff.

Staff awareness of the satisfaction of consumers is important because staff have considerable influence over the lives of consumers with whom they work. Accurate awareness may be an important indicator of how well staff and consumers communicate with each other and how well staff address the consumers’ basic needs. For example, if staff do not adequately understand the consumers’ needs for friendships and other social supports, the assistance that they provide in fostering social supports will likely be ineffective.

This study involved interviews with 245 pairs of consumers, who are dually diagnosed, and the staff members who were most familiar with them. The purpose of the study was to determine whether or not the staff members were aware of how satisfied the consumers were in several areas of daily living. Awareness was evident if the staff members’ responses to consumer satisfaction questions as representatives of the consumers were in agreement with the consumers’ responses.

Method

Research participants

The consumers included in this study were class members of the Thomas S. Case, a U.S. Federal Court lawsuit (Thomas S. v. Flaherty). This North Carolina lawsuit began on behalf of a 19 year-old patient in a state psychiatric hospital and developed into a statewide class action suit. The suit focused on adults who have mental retardation or who have been treated as such and had been inappropriately placed in state psychiatric hospitals in conditions that violated their Constitutional rights. Constitutional rights to safety, protection from harm, treatment under safe conditions, freedom from undue restraint, and minimally adequate habilitation or treatment, were pivotal to this case (Calhoun, Dudley, & Bradley, 1993). This case was decided in favor of the plaintiffs and is considered an important recent legal precedent for the right to habilitation in the least restrictive settings. The State of North Carolina was the defendant in this case, and as such, was under the oversight of the federal court as it moved toward full compliance with the court order. In January 1998, Judge Richard Voorhees ruled to end the court order because of significant improvements in state-supported services and significant gains in pertinent consumer outcomes.

Overall, the participants in this study consisted of 245 pairs of consumers and the staff members who were most acquainted with each of them. These 245 consumers represent the entire group of former class members who participated in the fourth wave of annual interviews with the Thomas S. Longitudinal Study in 1996-97. Wave 4 was the first time that a set of consumer satisfaction questions was asked of the staff person as well as the consumer.

Sixty-six percent of the 245 consumers were males. Fifty-six percent were white, 41 percent were African American, and the remaining three percent were Native Americans or Latinos. The consumers’ mean age was 42.6 years (SD = 11.8), ranging from 23 to 89 years. The majority were labeled with mild mental retardation (53 percent), 30 percent were labeled moderate, 13 percent severe or profound, and the remaining four percent either were not labeled or the level was unknown. Most of these consumers were reported by staff members to have mental illness, with 32 percent having “some” mental illness and 39 percent having “major” mental illness. These 245 consumers mostly lived in community group homes or apartments at the time of the interviews; only ten percent lived in state psychiatric facilities, state mental retardation centers, nursing facilities, or rest homes.

Virtually all of the 245 staff members interviewed were full-time employees (97 percent). Forty-two percent had a college degree or more education. Most had introductory training in mental illness (84 percent) and mental retardation (83 percent). The length of their work experience in the mental health/developmental disabilities field averaged 6.7 years (SD = 5.9) and their work experience at their present work site averaged 2.5 years (SD = 2.7). These staff members were asked how much they liked their job and this consumer on a ten point scale with “1” being not at all and 10 being “a lot”. Their responses to both questions were very high (mean score of 8.7 for their job and 9.1 for the consumer).

Procedure

Both the consumers and the staff members were interviewed separately, using 21 paired questions on the same topics. The questions focused on the consumers’ satisfaction with their living arrangements and roommates, the staff members who work closely with them, their jobs and work conditions, their friendships, and safety issues. In all cases, these pairs of questions were asked as similarly as possible. For example, the consumer was asked, “Do you like living here?” and the staff person was asked, “Does (consumer) like living here?” Not all-245 consumers responded to every question. For example, consumers who were employed only answered employment-related questions.

Table 1 describes an abbreviated version of the questions. The most favorable response is listed after each question, in parentheses, followed by the frequencies of when staff gave more favorable responses and when consumers gave more favorable responses. The most “favorable” response may be a debatable issue for some of the questions, depending upon one’s viewpoint.

A Wilcoxon Signed Ranks test was used to determine the extent of their disagreement on these 21 pairs of questions. When disagreement between these two groups was found to be statistically significant, a regression analysis was used to determine possible predictors of disagreement. Several staff and consumer characteristics were considered as independent variables in the regression equation including the consumer’s race, gender, level of mental retardation, degree of mental illness, and type of residence (institutional or community); and the staff person’s extent of training, formal education, length of employment in the mental health/developmental disabilities field and at the current site, and how much he/she liked the job and this consumer.

Results

Table 1 summarizes the consumers’ and staff members’ responses including their agreements and disagreements on each pair of questions. The results reveal considerable agreement on most questions. For example, agreement occurred 60 percent or more of the time on 17 of the 21 questions and 70 percent or more of the time on 13 of the questions.

Statistically significant disagreements

The consumers and staff members significantly disagreed with each other on five questions (p < .05). First, they disagreed about whether or not the consumers felt that their family visited enough (Question 3). Significantly more consumers were likely to feel that their family visited enough than staff realized. Second, consumers and staff disagreed on whether the consumers felt safe or afraid at their residence (Question 4). In this instance, significantly more consumers expressed fear than staff realized.

Third, they disagreed on whether or not the consumers felt that they spent enough time with their friends (Question 14), with significantly more consumers being satisfied than staff realized. Fourth, they disagreed on whether the consumers wanted to stay where they currently lived or move elsewhere (Question 21). Significantly more consumers were likely to want to move than staff realized.

Finally, they disagreed on the identification of the consumer’s best friend (Question 12). Choices of a best friend included three possible categories: a staff member, another consumer, or someone else. Sixty-one consumers identified a staff person as their best friend and 45 of the staff assigned to these 61 consumers also identified a staff person. Fifty-seven consumers identified another consumer as their best friend, and 31 of the 57 staff assigned to them agreed. Finally, 38 consumers identified someone other than a staff person or consumer as their best friend, and only 11 of their staff agreed. In addition, 24 consumers said that they had no best friend.

Predictor variables were found for disagreements for only one of the above five questions. The residential setting of the consumer (institutional or community) was found to be a predictor variable for why more consumers felt fearful than staff realized. In this case, if the consumer lived in an institution, the staff person was less likely to be aware of such fears {Exp (B) = 7.44, p = .04}. Otherwise, none of the 14 variables entered in the regression equation was found to be a predictor of disagreements for any of the above five questions.

Considerable disagreement was also evident on other questions as well, but there were not statistically significant differences in how consumers and staff disagreed. Instances of this were two employment questions (Questions 9 & 10) and whether other consumers at their residence were perceived to be mean or nice (Question 18).

Summary

Important implications are evident in these results, particularly in the instances in which there were significant disagreements. Satisfaction with family visits is one such instance. Why did more consumers report being satisfied with visits from family members than staff realized? Perhaps, some consumers were in denial about their need for their family while staff members were more in touch with their deeper dissatisfactions? Or possibly staff members were projecting their own concerns that family members should visit more often? Whatever the reasons for the disagreements, satisfaction with family visits may be an important topic to discuss more freely with some consumers.

Fears of consumers are another important concern, particularly because many consumers with a dual diagnosis have aggressive type behaviors (Dudley et al., 1999). Apparently, more consumers were afraid, of something or someone in their surroundings, than was known to staff. Further, this was more likely to occur if they lived in an institution. Living conditions in state hospitals, for example, have been found to be less desirable by consumers than community residences (e.g., Lehman, Possidente, & Hawker, 1986). Having fears can adversely influence satisfaction in many areas of daily living, and at least some fears may be based on real dangers. Therefore, more attention may need to be given by staff members to discovering and understanding the fears of many dually diagnosed consumers.

Disagreements were also evident in whether consumers wanted to stay where they were currently living or move somewhere else. Apparently, more consumers wanted to move than staff members realized. Wanting to move may be a topic of extreme importance to some consumers; in other instances it may be simply a desire for a change of “scenery” even when things are going fairly well. Nevertheless, more staff/consumer discussion may be beneficial on wishes to move elsewhere or leave a current residence.

Disagreements about the consumer’s best friend may deserve a different kind of response than the other questions in the study. Perhaps, disagreements in this case are desirable. Since we are preparing consumers to be more independent and self-determining, it may not be a good idea for staff members to know everything about the desires and needs of consumers assigned to them. Some aspects of the consumers’ lives, such as the identity of a best friend, may be better kept private. Thus, the considerable disagreement between consumers and staff in identifying their best friend could be viewed as a positive indicator.

Overall, this study is likely to have direct value for administrators and practitioners in the mental health/developmental disability field. The design of this study, in particular, may offer a useful exercise. Staff persons and consumers could respond to questions like those in Table 1 and then find out how much they are in disagreement. If there are disagreements, these may be good opportunities to open up fuller discussions on the needs and desires of consumers so that they can be better understood.

References:

Brown, C., Cosgrove, N., & DeSelm, T. (1997). Barriers interfering with life satisfaction for individuals with severe mental illness. Psychiatric Rehabilitation Journal, 20, 67-71.

Calhoun, M. L., Dudley, J., & Bradley, V. (1993). The Thomas S. case: Analysis of history, policy and implementation, November 1992- February 1993. (Thomas S. Longitudinal Study Monograph 1). Thomas S. Longitudinal Research Project, Department of Sociology, Anthropology, and Social Work, University of North Carolina, Charlotte, NC.

Dudley, J., Calhoun, M., Ahlgrim-Delzell, L., & Conroy, J. (1998). Measuring the consumer satisfaction of class members of a law suit. Journal of Intellectual Disability Research, 42, 199-207.

Dudley, J., M. Calhoun, & L. Delzell.(1999). Diverse diagnostic and behavioral patterns amongst people with a dual diagnosis. Journal of Intellectual Disability Research, 43, 70-79.

Lehman, A., Possidente, S., & Hawker, F. (1986). The quality of life of chronic patients in a state hospital and in community residences. Hospital and Community Psychiatry, 37, 901-907.

Lunsky, Y. & Benson, B. (1997). Reliability of ratings of consumers with mental retardation and their staff on multiple measures of social support. American Journal on Mental Retardation, 102, 280-284.

Sigelman, C., Budd, E., Spanhel, C., & Schoenrock, C. (1981). When in doubt, say yes: Acquiescence in interviews with mentally retarded persons. Mental Retardation, 19, 53-8.

Thomas S. v. Flaherty, 699F. Supp. 1178 (WDNC, 1988).

For further information:

James R. Dudley
University of NC at Charlotte
Charlotte, NC 28223
Email: jrdudley@email.uncc.edu