Service Users’ Perceptions About Their Hospital Admission Elicited by Service User–Researchers or by Clinicians

This study was conducted in three hospitals: St. Vincent’s University Hospital, which is a general hospital with a psychiatric ward; Newcastle Hospital, and St. John of God Hospital. These three hospitals receive admissions from three community mental health services in Ireland: Dublin South East mental health service and the Newcastle Mental Health Services in County Wicklow and the Cluain Mhuire Mental Health Service in County Dublin. These services cover a combined catchment area population of approximately 390,000. In addition to treating service users from the local catchment area, St. John of God Hospital is an independent psychiatric hospital and receives referrals on a national basis. We included service users who were admitted privately to St. John of God Hospital.

All individuals age 18 years and older who were involuntarily admitted to St. John of God Hospital between May 1, 2010, and June 30, 2011, St. Vincent’s University Hospital between August 1, 2010, and June 30, 2011, and Newcastle Hospital between November 1, 2010, and June 30, 2011, were eligible for the study. In order to have a comparable number of voluntarily admitted participants, the next voluntarily admitted service user after each involuntary admission was selected. We also included individuals who were admitted voluntarily but were subsequently detained involuntarily during the admission. According to the Irish Mental Health Act of 2001, individuals with a sole diagnosis of a personality disorder or substance misuse cannot be admitted involuntarily. Therefore, to have comparable samples, we excluded voluntarily admitted service users with either a sole diagnosis of a personality disorder or substance misuse. We also excluded individuals with dementia and intellectual disabilities; those with a first episode of psychosis were excluded because they were involved in another study protocol.

Written informed consent was obtained from all study participants, and ethics approval was granted at each of the three study centers. We did not offer compensation for participation in the interviews before discharge. We were not granted permission to obtain information about individuals who declined to participate in the study, and therefore we could not compare those who participated with those who declined.

Participants were randomly assigned to an interview with a service user–researcher or a clinician. Participants were interviewed once a discharge date was set, and interviews occurred in the days before discharge from the hospital. Each participant had a second interview with a clinician before discharge to confirm the diagnosis and measure the severity of symptoms. Participants were not blind to their randomization, because it was an important aspect of the study for participants to know that they were being interviewed by a service user–researcher or a clinician. All service user–researchers informed potential participants that they were a service user–researcher, and all clinicians informed them that they were a doctor or nurse. We did not accommodate requests by potential participants to be interviewed by the other type of interviewer. The interviewing clinician was not a member of the participant’s treatment team. Service user–researchers were involved in all stages of the study design, implementation, and dissemination of results.

To randomly assign participants, a list of the numbers 1 and 2 was generated in random sequence by a Web site (www.random.org); 1 indicated a service user–researcher and 2 indicated a clinician. This list was generated and kept by an individual who was independent of the study and based in a separate building. The list was concealed from both the researchers and principal investigator.

We used postrandomization consent (Zelen’s design [13]) because the delivery of consent is part of the difference between interviews conducted by service users and by clinicians. Had a service user–researcher or clinician elicited consent before randomization, this may have introduced bias because individuals may have refused consent if asked by a clinician but not by a service user–researcher, or vice versa. This procedure also allowed us to test a secondary outcome—that is, whether participation differed depending on whether consent was sought by a service user–researcher or a clinician.

Once a discharge date had been set, potential participants were informed by nursing staff on the ward that there was a researcher who would like to meet with them about a study. At this point, the potential participant was not informed whether the researcher was a service user–researcher or a clinician. If the potential participant agreed, a time was arranged for the meeting. Potential participants were provided with oral and written information about the study and were given an opportunity to ask questions and decide whether they wished to participate. There was no time constraint within which potential participants had to decide. Participants could withdraw consent at any stage, which was clearly stated in the letter of information and the consent form.

We used the MacArthur Admission Experience Survey (MacArthur AES) to determine the level of perceived coercion, perceived pressures, and procedural justice experienced by an individual on admission to the hospital (14). The primary outcome was level of perceived coercion, which is measured on a scale from 0 to 5, with higher scores indicating a higher level. The level of perceived coercion is obtained from five statements related to the individual’s experience of being admitted to the hospital. The five statements are: “I had more influence than anyone else on whether I came into hospital,” “I had a lot of control whether I went into hospital,” “I chose to come into hospital,” “I felt free to do what I wanted about coming into hospital,” and “It was my idea to come into hospital.” Answers of “true” are scored 0 and answers of “false” are scored 1. The level of perceived pressures is assessed with four yes-no questions and scored from 0 to 4, with higher scores indicating a higher level. Procedural justice reflects a person’s belief that others are acting out of genuine concern for him or her and that the person is being listened to and being treated respectfully and fairly. This is measured on a scale from 1 to 4, with higher scores indicating a lower level of experienced procedural justice (the person perceives that he or she was less fairly treated) (15).

To evaluate the perceived necessity of the admission, participants were asked, “Do you think it was necessary for you to have been admitted [involuntarily or voluntarily–depending on legal status] to hospital?” We accepted “yes,” “no,” or “don’t know” as answers.

We used the Client Satisfaction Questionnaire (CSQ-8), a self-report, eight-item instrument, to measure satisfaction with services received (16). The CSQ-8 uses a 4-point Likert scale (range 1–4), and a total score (range 8–32) is calculated from summing the scores on the eight items. There is no reliable or valid tool to assess whether an individual perceives the hospital admission to have been necessary. Therefore, we asked a closed question about whether the individual perceived the admission to have been necessary or unnecessary.

All individuals were assessed using the Structured Clinical Interview for DSM-IV by a trained clinician. This included the Global Assessment of Functioning, which ranges from 0–100, with 100 being the highest level of functioning (17). Positive and negative symptom severity was measured with the Scale for Assessment of Positive Symptoms and the Scale for Assessment of Negative Symptoms (18,19).

We measured depressive symptoms with the Beck Depression Inventory, and categories of depressive symptoms were classified as minimal depression (score of 0–9), mild depression (score of 10–18), moderate depression (score of 19–29), and severe depression (score of 30–63) (20). Insight was measured with the Birchwood Insight Scale, an eight-item scale on which scores range from 0 to 12, with higher scores indicating good insight (21).

Six clinicians (five physicians and one nurse) from the local mental health services involved in the study and four service user–researchers from SOURCE (Source of User Research, Collaboration and Expertise) conducted the interviews. SOURCE is a research group staffed by a clinical researcher and a group of individuals with past or current experiences of mental health difficulties. The main focus of the group is to use research methods to capture the lived experiences and perspectives of individuals who have experienced mental health difficulties, with the objective of improving mental health services.

Before the start of the project, SOURCE had affiliations with only one of the hospitals involved in the study—St. John of God Hospital. To our knowledge, research involving service user–researchers had not been conducted at the other two hospitals before this project. None of the hospitals employed service user–researchers in professional roles before the start of this project.

All interviewers underwent training in the MacArthur AES before the study. Interrater reliability was obtained for all service user–researchers and clinician researchers via interviews with eight inpatients. We developed videos with actors for any interviewers who joined the study after the start date and for retraining. The interrater reliability (kappa) at baseline ranged from .77 to 1.00 for the perceived coercion component, from .77 to 1.00 for perceived pressures, and from .66 to 1.00 for procedural justice.

Data were entered into a MS Access database and exported to PASW, version 18, for analysis. For the primary outcome, our a priori power calculation indicated that 102 participants (51 in each arm) were required to detect a difference of 1 in the perceived coercion scale of the MacArthur AES (measured 0–5) with 90% power at the 5% significance level. The null hypothesis states that there is no difference in the information elicited from interviews conducted by service user–researchers and by clinicians. We used chi square tests to determine associations for dichotomous variables and t tests to compare means on continuous variables between the two groups. The Mann-Whitney test was used to determine differences in nonparametric data.

Ideally, the randomization should have controlled for known and unknown confounders between the groups, and the characteristics in both groups should have been similar. However, differences between the groups were noted; individuals interviewed by service user–researchers were younger and more likely to be male and to have more depressive symptoms. These differences may have occurred by chance or as a result of the method of postrandomization consent. To control for these factors, we used analysis of covariance (ANCOVA) for the MacArthur AES and CSQ-8 because the dependent variable was quantitative and there was a combination of qualitative and quantitative explanatory variables. When depressive symptoms were added to the ANCOVA model in which perceived coercion was the dependent variable, a test indicated that the data violated the assumption of equality of error variation. Therefore, for perceived coercion, depressive symptoms were examined separately as a potential confounder and as a dichotomous variable (absent or mild versus moderate or severe depressive symptoms). We used binary logistic regression analysis to control for age, gender, and depressive symptoms for the secondary outcome of the perceived necessity of the admission. [Tables presenting the results of the ANCOVA analysis and the binary logistic regression are available online as a data supplement to this article.]

A total of 231 service users were eligible for study participation, of whom 70% (N=161) were interviewed. [A flowchart showing study recruitment and reasons for nonparticipation is included in the online data supplement.] Most of those who were not interviewed were discharged before randomization or before they were able to provide consent (N=23), or they declined to participate (N=31). Of the 161 participants, 48% (N=77) were interviewed by service user–researchers, and 52% (N=84) by clinicians. Those interviewed by service user–researchers were more likely to be male, to be younger, and to have more severe depressive symptoms. However, in terms of legal status, diagnosis, clinical characteristics, and the use of coercion, the groups were similar (Table 1). For the 161 patients in the full sample, the median duration of admission was 37 days (interquartile range 19–58 days)

Table 2 summarizes information elicited during the interviews by the service user–researchers and the clinicians. No differences were found by interviewer status in the reported level of perceived coercion on admission, either for individuals who were involuntarily admitted or for those who were voluntarily admitted. The findings were the same when the analysis controlled for age, gender, and depressive symptoms.

No differences were found by interviewer status in the levels of procedural justice, perceived pressures experienced on admission, and satisfaction with services, either for individuals who were involuntarily admitted or for those who were voluntarily admitted. The findings were the same when the analyses controlled for age, gender, and depressive symptoms.

Among the 161 service users, 118 (73%) viewed their admission to the hospital as necessary. No differences were found by interviewer status. Five (3%) individuals reported that they did not know whether their admission had been necessary, and three (2%) did not provide an answer. Of the 78 service users admitted involuntarily, 54% (N=42) viewed their admission as necessary, and no difference was found by interviewer status. Of the 83 service users admitted voluntarily, 92% (N=76) viewed their admission as necessary, and no difference was found by interviewer status. Results were the same when the analyses controlled for age, gender, and depressive symptoms.

Service users were more likely to decline to participate in the study when consent was sought by a service user–researcher compared with a clinician (24% versus 8%, χ²=9.1, df=1, p=.003). Similarly, the decline rates were higher for service user–researchers when consent was sought from involuntarily admitted service users (31% versus 14%, χ²=3.8, df=1, p=.05) and those who were voluntarily admitted (14% versus 2%, χ²=4.65, df=1, p=.05).

This study found no differences in the information elicited from service users about the level of perceived coercion, perceived pressures on admission, procedural justice, satisfaction with services, and perceived necessity of the admission when the interviews were conducted by service user–researchers or by clinician-researchers. Eligible individuals were less likely to consent to participate in the study if consent was sought by a service user–researcher.

The results are consistent with those of Rose and colleagues (10), who found no difference in service users’ reports of perceived coercion on admission in interviews conducted by three types of interviewers—service user–researchers who did or did not disclose their service user status and researchers who were not service users. Rose and colleagues sought to determine whether under certain conditions (status disclosed or not disclosed) service user–researchers would obtain differing reports, and the study reported here built on those findings by showing that service users’ reports did not differ when interviews were conducted by service user–researchers or by clinicians—a finding that was consistent for both voluntarily and involuntarily admitted service users.

This study found that 59% of service users viewed their involuntary admission as necessary. For involuntarily admitted service users, studies have reported a range of proportions—from 34% in the United Kingdom to 72% in Ireland (2,7). In this study, we interviewed individuals before discharge, which may explain the more positive views; the interviews in the large study in the United Kingdom were conducted on admission. Individuals’ views about their involuntary admission have been shown to change over time (22,23).

A unique aspect of this study is that the information elicited by service user–researchers was compared with information elicited by clinicians, and not simply with information elicited by researchers who were not service users. This finding has clinical implications in light of previous findings indicating that most service users have positive views of their involuntary admission after they are discharged (4). Some clinicians may be concerned that a retrospective positive description of a practice that deprived an individual of his or her civil liberties might result from response bias (7). However, if we assume that service users provide a reliable account of their admission to service user–researchers, then clinicians can have confidence in positive accounts provided to other types of interviewers. However, it must be noted that the study was designed to detect a difference in information elicited from interviews conducted by service user–researchers and by clinicians, which did not enable us to test whether the information obtained from the two types of interviewer was the same. Because most service users gave positive accounts of their admission, it may be that discharge is an opportune time to engage individuals in longer-term treatment if appropriate.

For other researchers, our finding of no differences in elicited information by interviewer status indicates that studies conducted by clinicians and service user–researchers can be compared directly or analyzed together in meta-analyses.

The finding that participants were less likely to provide consent to service user–researchers has several possible explanations that would require further qualitative study. However, there are methodological and ethical difficulties involved in studying a subgroup of individuals who do not wish to participate in research. One possible explanation is that participants were reassured by their knowledge that clinicians are bound by confidentiality, even though the letter of information and consent form stated that all interviews would be confidential. In a qualitative study that examined service users’ experience of being interviewed by another service user, the interviewees reported many positive aspects of the interviews, but confidentiality and insecurity were concerns (24). A method of formalizing the confidentiality of future interviews conducted by service user–researchers could be considered.

Alternatively, potential participants may have felt more comfortable declining when consent was sought by a service user–researcher, and the 24% decline rate may be a more accurate reflection of the proportion of individuals who would prefer not to participate in research. The low decline rate of 8% when consent was sought by clinicians may have resulted from a feeling of obligation to participate. In the EUNOMIA study of coercion in psychiatry in European countries, 26% of eligible individuals declined to participate; proportions ranged from 5% to 54% at the various European sites (25). Therefore, the decline rate for the service user–researchers was comparable with rates in other studies, and our finding of the 8% decline rate for clinicians may be particularly low. It has been suggested that consent sought by a clinician is associated with higher levels of perceived coercion to participate in research (26).

One limitation of the study is that we did not include individuals with a first episode of psychosis, which may have introduced bias to the sample. Another limitation is that service users’ views of their admissions are complex, and qualitative information provided during the interview was not captured. A further limitation is that the interviews were conducted in the hospital and before discharge; research has shown that both of these interview characteristics are associated with service users’ more positive accounts (8). Strengths of this study include an adequately powered sample size and inclusion of both voluntarily and involuntarily admitted service users. Because the sample was drawn from consecutive admissions in three large catchment areas and at a private hospital, our results are likely to be generalizable to other mental health services.