Assessing the Subjective Experience of Being a Participant in Psychiatric Research
Publication: American Journal of Psychiatry
Abstract
OBJECTIVE: To address both clinical and ethical concerns in psychiatric research, the study assessed the subjective experience of being a participant in a feasibility study of outcome in long-term psychodynamic psychotherapy and psychoanalysis. METHOD: A questionnaire assessing positive and negative reactions to three typical research methodologies (self-report questionnaires, structured diagnostic interviews, and tape-recording of sessions) was administered to 23 patient-therapist pairs. RESULTS: Patients reported that questionnaires and interviews were slightly to moderately helpful in promoting self-realization and facilitating therapy, and not at all to slightly intrusive and disruptive. Adjustment to audiotaping of sessions was rapid (within two sessions). Therapists significantly overestimated the negative effects and underestimated the positive benefit patients reported from participating in research. CONCLUSIONS: Traditional objections to research in dynamic psychotherapy on the grounds that patients experience research procedures as significantly intrusive and disruptive appear to be unfounded.
In the context of renewed attention to ethical and regulatory issues in human subjects research, it is striking that few data from treatment research subjects are available to inform this discussion (1). Such data are necessary for developing guidelines that ensure both the protection of human subjects and the integrity of research methods while avoiding unnecessary and burdensome procedural obstacles to clinical studies.
In particular, since clinical trials to study the efficacy or effectiveness of a treatment depend on recruitment of large numbers of patient participants (2, 3), the perceptions of both potential subjects and practicing clinicians toward research may be critically important to progress in clinical science. For example, many therapists believe that the evaluation and assessment procedures necessary to conduct methodologically rigorous outcome research distort the treatment process. To the extent this is true, generalization of results would be limited. If such views are unfounded, however, they represent a resistance to research and a major obstruction of the advancement of knowledge.
To address these issues, we developed a study in which research patients and clinicians were systematically surveyed to assess their experience of data collection procedures. The primary study was an empirical examination of the feasibility of studying outcome in our long-term psychodynamic psychotherapy and psychoanalysis clinics (4, 5).
In a search of the literature, we could identify no instrument that systematically assessed patients’ experiences in treatment research. A survey questionnaire was therefore developed in consultation with psychodynamic clinicians to assess and quantify commonly expressed concerns about research participation and its purported effect on treatment.
Method
The primary study, described elsewhere (4, 5), was developed to assess the feasibility of studying outcome in long-term psychodynamic and psychoanalytic therapies. Patients were treated in twice-weekly psychodynamic therapy with residents in psychiatry or in psychoanalytic treatment with candidate analysts. Research procedures included structured diagnostic interviews with the Structured Clinical Interview for DSM-III-R (axis I and II) (6) and the Social Adjustment Scale (7), use of 10 self-report forms (4) requiring approximately 1.5 hours to complete, and audiotaping of five consecutive therapy sessions. The baseline evaluation was repeated after 1 year of treatment.
The survey questionnaire we developed, entitled Assessing the Impact of Research on Subjects, is a 14-item (baseline evaluation) or 19-item (final evaluation) self-report questionnaire that uses a Likert-type scale to assess positive and negative emotional and cognitive appraisals of the experiences of undergoing a structured interview, completing questionnaires, and having sessions audiotaped. Positive items asked whether questionnaires and structured interviews promoted self-realization or facilitated the treatment. Negative items asked whether questionnaires and interviews seemed disruptive or intrusive. Audiotaping items asked about difficulty adjusting to audiotaping. The therapists’ questionnaire assessed the therapist’s perception of the patient’s experience and used wording that was identical to the patients’ version wherever possible.
The survey questionnaire was mailed with an addressed, stamped envelope to the patient and corresponding therapist within 1 week of the completion of the fifth audiotaped session at baseline, and within 1 week of the completion of structured interviews and self-report questionnaires at the 1-year evaluation. Institutional review board approval and written informed consent were obtained, and no monetary reimbursement was provided to patients.
Repeated measures analysis of variance or paired t tests, as appropriate, were used to investigate whether there were group differences at baseline and at 1 year, to assess for change over time, and to look for interaction effects.
Results
Twenty-three of 36 patients (63.9%) who were approached agreed to participate in the feasibility study, and all patients remaining in treatment elected to continue in the study at 1 year (N=15). Nineteen of the 23 patients (82.6%) were female, the mean age was 27.4 years (SD=7.9), and 18 (78.3%) had never married. All had at least some college education. Sixteen (69.6%) were Caucasian, three (13.0%) Asian, two (8.7%) black, one (4.3%) Hispanic, and one (4.3%) East Indian. The majority of patients (N=15, 65.2%) met lifetime criteria for an affective or anxiety disorder. There were no differences on demographic or survey variables between those who dropped out of treatment (N=8) and those who remained the full 1-year period (N=15).
At baseline, patients rated both self-report questionnaires and structured psychiatric interviews as slightly to moderately helpful in promoting new insight about themselves and slightly to moderately helpful in facilitating therapy (Table 1). Questionnaires and interviews were rated as not at all to slightly intrusive and disruptive and, overall, as a slightly negative experience (Table 1). Patients rated the research as only slightly to moderately unnecessary or irrelevant to treatment. Adjustment to audiotaping of sessions occurred within two sessions and very soon after sessions began (Table 1).
There was no statistically significant change between baseline and 1-year ratings for patients or therapists on any variable.
Therapists’ ratings of the overall positive impact of research on patients were significantly lower than the patients’ self-report ratings of the positive impact of research. The same pattern was seen on ratings of whether and to what degree questionnaires promoted self-realization and facilitated therapy (Table 1). Patients’ assessments of whether structured diagnostic interviews were disruptive and intrusive were significantly less negative than therapists’ assessments (Table 1). Patients and therapists did not differ on their ratings of the experience of tape-recording the sessions (Table 1).
On a question about the informed consent process, which asked whether patients felt well informed before participating, most patients (nine of 15) reported feeling they were well informed, three were not sure, and three answered “no” to this question. However, nearly all patients (14 of 15) who completed the study stated they would definitely or probably participate if they had it to do over again.
Discussion
Our main finding was that patients enrolled in a psychodynamic psychotherapy research study reported minimal negative impact from participating in research, and a slight to moderate positive impact. The research procedures in the study (questionnaires, interviews, and tape recording of sessions) are commonly used in clinical trials. Furthermore, clinicians underestimated the positive benefit of research participation to patients and overestimated the intrusive and disruptive aspects of the research, compared to patients’ ratings. No patients who continued in treatment dropped out of the research, and the dropout rate was comparable to that observed in other clinical trials of psychosocial or medication treatments (eight of 23 patients, 34.8%). Our clinical trial findings complement those from a large descriptive study of psychological trauma in which almost all participants reported minimal negative reactions to the research procedures and found participation somewhat useful (8).
Confidence in the results of this study is limited by two factors: 1) a relatively small sample size, which can be addressed in future studies, and 2) the use of a new instrument that lacks established test-retest reliability or construct validity. The sample size limits interpretation of nonsignificant findings in particular.
This study pertains to subjects who volunteer for research participation. The potential experience of those who declined to participate is unknown, but irrelevant, since informed consent is designed to allow individuals who might find research too burdensome or intrusive to decline participation. The fact that, at the conclusion of the study, some patients reported that they did not feel well informed can have several interpretations. Patients may not have fully understood explanations of the study during the informed consent procedures, or the informed consent procedures may not have been adequate. Alternatively, verbal descriptions may have a limited capacity to fully communicate in advance such a complex experience as research participation. However, the willingness of nearly all patients to participate again shows that those aspects of research that might be experienced as burdensome were well communicated through the informed consent procedures, and provides reassurance about the ethical integrity of these procedures.
The results of this study have several implications. First, we found that it was feasible to study research subjects’ experiences. Such data are critical in informing discussions about research ethics. Second, patients reported minimal negative reactions, despite participating in a fairly intensive assessment that included audiotaping of sessions and approximately 2.5 hours of diagnostic interviews. Third, patients reported a positive benefit that was due to the research methodologies themselves, a possibility that is rarely mentioned in discussions of research practices. Fourth, clinicians significantly underestimated the positive benefits of research participation for patients and overestimated the negative aspects. This finding suggests that therapists’ assessments of their patients’ experience in research were, not surprisingly, influenced by their own attitudes. The belief held among some therapists that research is necessarily intrusive and harmful was not supported by our data.
In fact, a few prior reports on the impact of research participation on clinical outcome found that its effects ranged from negligible to significantly positive (9–11). These findings are especially important, as it is our impression that resistance to research based on such objections has impeded progress specifically in the study of psychodynamic psychotherapies.
Footnote
Received April 12, 1999; revisions received Dec. 1, 1999 and June 27, 2000; accepted July 27, 2000. From the New York State Psychiatric Institute, Columbia University College of Physicians and Surgeons, New York. Address reprint requests to Dr. Marshall, Director of Trauma Studies, Anxiety Disorders Clinic (Unit 69), New York State Psychiatric Institute, 1051 Riverside Drive, New York, NY 10032; [email protected] (e-mail). Supported in part by NIMH grant MH-01412 to Dr. Marshall and a grant from the Dorgan Fund, Department of Psychiatry, Columbia University College of Physicians and Surgeons.
References
1.
Moreno J, Caplan AL, Wolpe PR: Updating protections for human subjects involved in research: Project on Informed Consent, Human Research Ethics Group. JAMA 1998; 280:1951–1958
2.
Lundberg GD, Wennberg JE: Quality of care: a call for papers for the annual coordinated theme issues of the AMA Journals. Arch Gen Psychiatry 1996; 53:1085
3.
Wells KB: Treatment research at the crossroads: the scientific interface of clinical trials and effectiveness research. Am J Psychiatry 1999; 156:5–10
4.
Vaughan SC, Marshall RD, MacKinnon RA, Roose SP: Current psychotherapy research methodology applied to psychoanalysis: a feasibility study. J Psychother Prac Res 1994; 3:334–340
5.
Marshall RD, Vaughan SC, MacKinnon RA, Mellman, L, Roose SP: Assessing outcome in psychoanalysis and long-term psychodynamic psychotherapy. J Am Acad Psychoanal 1996; 24:575–604
6.
Spitzer RL, Williams JBW, Gibbon M: Structured Clinical Interview for DSM-III-R (SCID). New York, New York State Psychiatric Institute, Biometrics Research, 1987
7.
Weissman MM, Sholomskas D, John K: The assessment of social adjustment: an update. Arch Gen Psychiatry 1981; 31:1250–1258
8.
Newman E, Walker EA, Gefland A: Assessing the ethical costs and benefits of trauma-focused research. Gen Hosp Psychiatry 1999; 21:187–196
9.
Ben-Arie O, Koch A, Welman M, Teggin AF: The effect of research on readmission to a psychiatric hospital. Br J Psychiatry 1990; 156:37–39
10.
Carroll RS, Miller A, Ross B, Simpson GM: Research as an impetus to improved treatment. Arch Gen Psychiatry 1980; 37:377–380
11.
Kocsis JH, Frances A, Kalman TP, Shear MK: The effect of psychobiological research on treatment outcome: a controlled study. Arch Gen Psychiatry 1981; 38:511–515
Information & Authors
Information
Published In
History
Published online: 1 February 2001
Published in print: February 2001
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).