Spanish Validation of the Problem Behaviors Assessment–Short (PBA-s) for Huntington’s Disease
Publication: The Journal of Neuropsychiatry and Clinical Neurosciences
Abstract
A prospective, observational multicenter study was carried out assessing neuropsychiatric symptoms in a sample of 117 subjects in order to validate the Spanish version of the Problem Behaviors Assessment-Short (PBA-s). The psychometric properties of this version were analyzed. Inter- and intra-rater reliability were good: the mean weighted Cohen’s kappa was 0.90 for severity scores and 0.93 for frequency scores. Four factors accounting for 56% of the total variance were identified after an exploratory factor analysis: apathy, irritability, depression, and perseveration. The PBA-s correlates strongly with the Neuropsychiatric Inventory, demonstrating its accuracy for assessing neuropsychiatric symptoms in patients with Huntington’s disease.
Huntington’s disease (HD) is a rare neurodegenerative disorder with an autosomal dominant inheritance that usually affects people in their forties. It progresses toward a severe functional decline and dementia, finally causing death, frequently due to comorbid complications such as respiratory infections and severe trauma after a fall.1 HD manifestations include motor disturbances, cognitive impairment, and neuropsychiatric symptoms. Classically, motor disturbances defined the onset of the disease, even though several studies have shown the presence of cognitive and behavioral changes prior to the onset of a movement disorder.2–4
Neuropsychiatric symptoms are frequent in HD, as referenced in many studies.5–7 A recent study8 of the European Huntington’s Disease Network Behavioral Phenotype Working Group using the behavioral section of the Unified Huntington’s Disease Rating Scale (UHDRS)9 confirmed that depression, irritability/aggression, obsessive-compulsive behaviors (OCBs), and apathy are highly prevalent neuropsychiatric symptoms in HD. They concluded that some of these symptoms are also distressing and are often unresolved by treatment in all stages of HD, with apathy being the key neuropsychiatric symptom occurring most often in advanced HD stages. These symptoms lead to important impairments in the caregivers’ quality of life.10
Despite its high prevalence, some neuropsychiatric symptoms have often been underappreciated, underdiagnosed, and undertreated.8 Neuropsychiatric symptoms also result in impairment in quality of life and are a considerable burden on health care systems, so assessing them accurately is very important to provide the best care to HD patients.11
However, there have been few scales that were specifically designed to address psychiatric disturbances in HD patients to date. The most relevant ones are the UHDRS Behavioral Assessment, the Problem Behaviors Assessment for HD (PBA-HD),5 and its short version, the Problem Behaviors Assessment-Short (PBA-s).12
The UHDRS was developed in 1996 to provide a uniform assessment of the clinical features and course of HD. The behavior component of the UHDRS is an 11-item scale that was used as the global psychiatric measure in most HD studies.9
The PBA-HD is a semistructured behavioral interview designed for rating the presence, frequency, and severity of behavioral abnormalities in patients with Huntington’s disease.5 The PBA-HD is a 40-item instrument intended for use by trained psychiatrists with experience in the assessment of patients with neuropsychiatric disorders. The scale was developed via the compilation of psychiatric symptoms that patients and relatives frequently reported or those that are often referred to in the literature due to clinical interest in the field of dementia syndromes affecting frontal lobes.5 The instrument was developed in two phases. First, a five-point (0–4) scale was included for rating the severity and frequency of each symptom recorded as present or absent for the period since the patient’s last visit to the clinic, as done in the UHDRS. Second, strict scoring criteria for rating the severity of each symptom were included, and frequency criteria for every symptom were also defined to improve reliability. In order to maintain compatibility with the UHDRS Behavioral Scale, the 4-week period immediately prior to the interview was chosen for the assessment. Relative to its psychometric properties, the PBA-HD showed good feasibility; in their original study, the authors referred to good reliability (inter-rater mean weighted kappa: 0.86; test-retest mean weighted kappa: 0.94).5 Kingma et al.13 created a Dutch translation of the PBA-HD in 2008. This version was able to replicate the previous results: an interrater reliability of 0.82 for severity scores and 0.73 for frequency scores in a larger sample than that used in Craufurd et al.’s original study.
The PBA-s,12 an 11-item semistructured clinical interview, was developed by the EHDN Behavioral Phenotype Working Group for use in the REGISTRY study,12 where a shorter scale was necessary. The PBA-s is an instrument that evaluates severity and frequency of these symptoms: depressed mood, suicidal ideation, anxiety, irritability, angry/aggressive behavior, apathy, perseveration, obsessive-compulsive behavior, paranoid thinking, hallucinations, and disoriented behavior. To improve reliability, suggested prompts are offered to the interviewers, and criteria for rating frequency and severity are provided. The frequency scores range from 0 (symptom absent) to 4 (present all the time). The severity scores are defined independently for each item, ranging from 0 (symptom absent) to 4 (symptom causing severe problems). Severity and frequency scores are multiplied to obtain a total score for each item. The final score is rated considering all available information: that given by the patient and relatives, as well as the interviewer’s own observations of the patient’s behavior. Patients and informants are asked to identify the presence or absence of the symptoms during the last month, but in order not to exclude important symptoms that could have occurred during the last year (the recommended period to apply the scale within the longitudinal studies that are usually performed in the HD population), a “worst” score is added to set the severity of the symptom during that period of time.
Recently, the psychometric proprieties of the PBA-s have been reported using the data from two large studies: TRACK-HD and REGISTRY.14 The reliability study included data from 410 PBA-s video-recorded interviews, conducted with a total of 308 participants. The authors concluded that the PBA-s demonstrated good reliability. The mean kappa was 0.74 for severity and 0.76 for frequency scores, whereas the weighted kappa (allowing scores to differ by 1 point) was 0.94 for severity and 0.92 for frequency scores.
McNally et al.15 conducted a study that aimed to determine the validity of the PBA-s for Huntington’s disease as a clinical outcome measure using Rasch analysis. This study highlighted important limitations of the PBA-s to achieve this goal. Some amendments could lead to a better construct validity: PBA-s severity items could be combined to form a valid total score measuring a unidimensional construct, with reduction to a four-point scale. Attempts to achieve fit to the Rasch model for the PBA-s frequency score were more problematic.
Due to the difficulty in applying the diagnostic criteria for major psychiatric conditions, the PBA-s may be a good alternative to approach the psychological and behavioral manifestations of HD. It is recommended that the PBA-s be used in all HD studies where a behavioral assessment is needed as a comprehensive screen for the most common neuropsychiatric symptoms in HD. REGISTRY, a Europe-wide study run by the EURO-HD Network, and Enroll,16 a worldwide HD study, incorporated the PBA-s as a core assessment. Spain and other Spanish-speaking countries participated in these studies, so validating the PBA-s was absolutely necessary.
Methods
Design and Participants
A prospective observational multicenter study was carried out between January 2013 and December 2014. One hundred seventeen participants were recruited with the same criteria from five Spanish centers within the European Huntington Disease Network: 80 (68.4%) from the Hospital Mare de Déu de la Mercè (Barcelona), 18 (15.38%) from the Hospital Ramón y Cajal (Madrid), 11 (9.4%) from the Hospital Universitario (Burgos), five (4.27%) from the Hospital Clinic (Barcelona), and three (2.56%) from the Hospital Virgen de Arriaxaca (Murcia). The Ethics Committee, CEIC Hermanas Hospitalarias (Barcelona, Spain), approved the study protocol in accordance with the Declaration of Helsinki of 1964, revised in Edinburgh in 2000. Written informed consent of the participants was obtained.
The group of participants was comprised of 117 subjects (104 outpatients and 13 inpatients). The distribution within the inpatient group was nine patients admitted to the neuropsychiatry hospitalization ward and four patients admitted to the neuropsychiatry day care hospital. The mean stay of these inpatients was 148 days. Patients were categorized into five groups: people at risk for HD, premanifest expansion mutation carriers, manifest HD patients, control expansion negative, and control without a family history of HD, following the structure of the REGISTRY study. In our sample, 77 participants were manifest HD patients, nine people were at risk, 12 were asymptomatic carriers, and 19 were controls. Informants were coded following the instructions of the author as follows: spouse or partner (N=34), parent (N=12), sibling (N=10), child (N=5), other relative (N=2), friend or neighbor (N=1), professional care worker (N=10), other (N=3), no informant-subject came alone (N=40).
Measures
A sociodemographic data sheet and three questionnaires were used: PBA-s, UHDRS Total Functional Capacity (TFC), and Neuropsychiatric Inventory (NPI).17 The NPI was administered the same day but after the PBA-s, with the informant alone; thus, this instrument is based on responses given by a well-informed caregiver. The sociodemographic data sheet provided information about many characteristics of the group, including age, gender, educational level, and other variables, such as CAG repetition number when suitable, disease stage, and rater category.
The TFC is an ordinal component of the UHDRS. Scores on the TFC represent five stages in the neurodegenerative disease process. Lower scores represent greater functional impairment: stage I represents scores of 13–11; stage II, scores of 10–7; stage III, scores of 6–3; stage IV, scores of 2–1; and stage V, a score of 0.
The NPI is an instrument designed to capture most of the more frequent psychiatric and behavioral symptoms in dementia, with a similar structure to the PBA-s. Twelve items are included: delusions, hallucinations, agitation, dysphoria, anxiety, apathy, irritability, euphoria, disinhibition, aberrant motor behavior, nighttime behavior disturbances, and appetite and eating abnormalities. The frequency and severity of every item are measured on the basis of scripted questions administered to the patient’s caregiver, and a total score of every single item is obtained by multiplying the scores of the different domains.
Procedure
The validation process was carried out following the recommended methodological approaches for translation, adaptation, and cross-cultural validation of research instruments.18 The process began by first making a direct translation, which was done by professionals with a suitable level of English and psychopathological expertise and who were experienced in Huntington’s disease. Each translator performed his or her translation independently, and, after several meetings, a consensus version was obtained. Psychiatrists and neuropsychologists from other units of the hospital, unrelated to HD, reviewed the first version in order to evaluate its comprehensibility and suitability. A pilot study was carried out using a limited number of subjects, and this was designed to evaluate the instrument’s feasibility (evaluate the instructions, items, and response format clarity as well as application time). Afterward, and once this preliminary version was reviewed, a bilingual translator created a back translation to compare the differences between the translated version and the original text.
To evaluate reliability, we videotaped the interview of the first 30 patients enrolled consecutively and who gave informed consent to perform it. One rater (JR-I) re-evaluated his own interview after one month to measure intra-rater reliability, and two different raters (MF and CM) evaluated their interviews independently to evaluate inter-rater reliability. Concurrent validity was also measured. We chose to compare the results of the PBA-s Spanish version with those obtained by simultaneously applying the NPI.
Statistical Analysis
Descriptive statistics of the reference group are reported. Internal consistency was tested using the Cronbach’s α coefficient.19 Intra- and inter-rater reliability were assessed using Cohen’s kappa (κ).20 Cohen’s kappa takes into account disagreement between the two raters, but not the degree of disagreement. This is especially relevant when the ratings are ordered, and therefore, the weighted kappa (κw) was also calculated.21 Weighted kappa allows scores to differ by 1 point. A Spearman correlation coefficient was computed between the total scores of the PBA-s and NPI (N=46), to test the concurrent validity of the scale, and between the PBA-s and the TFC scores. This statistical analysis was performed using the IBM SPSS Statistics (version 23.0).
In order to evaluate the factor structure of the severity of symptoms, we carried out a factor analysis on the severity scores (Table 1). Specifically, we applied the factanal function from the R statistical package22 to perform a maximum-likelihood factor analysis with varimax rotation. To avoid confounding effects from both unrelated controls and former patients with negative test outcomes, these two subgroups were excluded from the analysis (leaving an N=98). From the pool of symptoms, “delusions” and “hallucinations” were discarded, as less than 10% of individuals scored nonnull values for the severity of both symptoms.
| Symptom | Category | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| At-Risk (N=9) | Asymptomatic Carrier (N=12) | Manifest HD (N=77) | Control Expansion Negative (N=6) | Control Without HD Family History (N=13) | ||||||
| Present | Moderate/Severe | Present | Moderate/Severe | Present | Moderate/Severe | Present | Moderate/Severe | Present | Moderate/Severe | |
| Last month | ||||||||||
| Apathy | 2 (22.2) | 0 | 3 (25) | 0 | 52 (67.5) | 31 (59.6) | 1 (16.7) | 0 | 2 (15.4) | 0 |
| Anxiety | 4 (44.4) | 3 (75) | 8 (66) | 2 (25) | 42 (54.5) | 16 (38.1) | 4 (66.7) | 1 (25) | 10 (76.9) | 5 (50) |
| Perseverative thinking/behavior | 2 (22.2) | 0 | 1 (8.3) | 0 | 26 (36.8) | 21 (27.3) | 1 (16.7) | 0 | 2 (15.4) | 0 |
| OCB | 1 (11.1) | 0 | 0 | 0 | 18 (23) | 3 (16.7) | 1 (16.7) | 0 | 1 (7.7) | 0 |
| Depressed mood | 4 (44.4) | 0 | 7 (58.3) | 1 (14.3) | 29 (37.7) | 7 (9.1) | 3 (50) | 1 (33.3) | 7 (53.8) | 1 (14.3) |
| Suicidal ideation | 0 | 0 | 1 (8.3) | 0 | 9 (11.7) | 3 (33.3) | 1 (16.7) | 0 | 2 (15.4) | 0 |
| Irritability | 4 (44.4) | 1 (25) | 6 (50) | 0 | 42 (54.5) | 15 (35.7) | 4 (66.7) | 0 | 9 (69.2) | 0 |
| Angry/aggressive behavior | 1 (11.1) | 0 | 2 (16.7) | 0 | 21 (27.3) | 14 (66.7) | 4 (66.7) | 0 | 1 (7.7) | 0 |
| Delusions/paranoid thinking | 0 | 0 | 0 | 0 | 6 (7.8) | 3 (50) | 0 | 0 | 0 | 0 |
| Hallucinations | 0 | 0 | 0 | 0 | 4 (5.2) | 2 (50) | 0 | 0 | 0 | 0 |
| Disoriented behavior | 0 | 0 | 0 | 0 | 32 (41.6) | 5 (15.6) | 0 | 0 | 0 | 0 |
| Last year | ||||||||||
| Apathy | 2 (22) | 1 (50) | 3 (33.3) | 1 (25) | 53 (68.8) | 33 (62.3) | 1 (16.7) | 0 | 2 (15.4) | 0 |
| Anxiety | 4 (44.4) | 3 (75) | 9 (75) | 4 (44.4) | 47 (61) | 27 (57.4) | 4 (66.7) | 2 (50) | 10 (76.9) | 6 (60) |
| Perseverative thinking/behavior | 2 (22.2) | 1 (50) | 1 (8.3) | 0 | 47 (61) | 25 (53.2) | 1 (16.7) | 0 | 2 (15.4) | 0 |
| OCB | 1 (11.1) | 0 | 0 | 0 | 17 (22.1) | 4 (23.5) | 1 (16.7) | 0 | 1 (7.7) | 0 |
| Depressed mood | 4 (44.4) | 0 | 7 (58.3) | 1 (8.3) | 31 (40.3) | 14 (45.2) | 3 (50) | 1 (33.3) | 7 (53.8) | 2 (28.6) |
| Suicidal ideation | 0 | 0 | 1 (8.3) | 0 | 9 (11.7) | 3 (33.3) | 1 (16.7) | 0 | 2 (15.4) | 0 |
| Irritability | 4 (44.4) | 1 (11.1) | 6 (50) | 1 (8.3) | 46 (59.7) | 25 (54.3) | 4 (66.7) | 1 (25) | 9 (69.2) | 1 (11.1) |
| Angry/aggressive behavior | 1 (11.1) | 0 | 1 (8.3) | 0 | 29 (37.7) | 22 (75.9) | 2 (33.3) | 0 | 2 (15.4) | 0 |
| Delusions/paranoid thinking | 0 | 0 | 0 | 0 | 9 (11.7) | 7 (77.8) | 0 | 0 | 0 | 0 |
| Hallucinations | 0 | 0 | 0 | 0 | 4 (5.2) | 2 (50) | 0 | 0 | 0 | 0 |
| Disoriented behavior | 0 | 0 | 0 | 0 | 31 (40.3) | 5 (16.1) | 0 | 0 | 0 | 0 |
a
N (% within present); HD: Huntington’s disease; OCB: obsessive-compulsive behavior.
Results
Demographical and Clinical Data
All relevant demographical and clinical baseline data are highlighted in Table 2. Ninety-two subjects (78.6%) were receiving psychiatric medication, most of them in combination therapy regimen: 51 (43.6%) were taking antidepressants, 46 (39.3%) antipsychotics, 38 (32.5%) anxiolytics, and 11 (9.4%) antiepileptic drugs. Twenty-nine patients (24.8%) were on specific treatment with antichorea therapy (tetrabenazine and/or amantadine).
| Characteristic | Category | ||||
|---|---|---|---|---|---|
| At-Risk (N=9) | Asymptomatic Carrier (N=12) | Manifest HD (N=77) | Control Expansion Negative (N=6) | Control Without HD Family History (N=13) | |
| Age (years) | 37.0 (13.71) | 38.2 (8.44) | 51.1 (12.22) | 44.1 (16.33) | 61.0 (8.73) |
| Gender (male/female) | 6/3 | 2/10 | 38/39 | 3/3 | 2/11 |
| Larger allele | — | 43.5 (2.46) | 44.6 (4.07) | 22.6 (5.27) | |
| TFC (N [%]) | |||||
| Stage I | 9 (100) | 12 (100) | 14 (18.2) | — | — |
| Stage II | — | — | 18 (23.4) | — | — |
| Stage III | — | — | 21 (27.3) | — | — |
| Stage IV | — | — | 21 (27.3) | — | — |
| Stage V | — | — | 3 (3.9) | — | — |
| Educational level | |||||
| ISCED 0 | — | — | 2 (2.6) | — | — |
| ISCED 1 | — | — | 19 (24.7) | 2 (33.3) | 6 (46.2) |
| ISCED 2 | 2 (22.2) | 2 (16.7) | 8 (10.4) | 1 (16.7) | 1 (7.7) |
| ISCED 3 | 1 (11.1) | 6 (50) | 16 (20.8) | 3 (50.0) | 1 (7.7) |
| ISCED 4 | 1 (11.1) | — | 6 (7.8) | — | — |
| ISCED 5 | 3 (33.3) | 3 (25) | 20 (26.0) | — | 2 (15.4) |
| ISCED 6 | 1 (11.1) | 1 (8.3) | 4 (5.2) | — | — |
| Unknown | 1 (11.1) | — | 2 (2.6) | — | 3 (23.1) |
a
HD: Huntington’s disease; TFC: total functional capacity; ISCED: International Standard Classification of Education.
In order to analyze the clinical characteristics of the neuropsychiatric symptomatology in our sample, a symptom with a 0 score is considered as absent and a 1 score as slight or questionable, whereas scores ≥2 are considered clinically relevant. Severity of the symptoms was classified as mild (scores 1 and 2) or moderate–severe (scores 3 and 4). In the tables below, neuropsychiatric symptoms of every participant category are shown, either in the last month or the last year. Results for every single present symptom are shown. In order to focus on the clinical relevance of the symptoms, the absolute number and the percentage of those considered as moderate–severe within each present symptom (scores 3 and 4) are shown.
Focusing on the neuropsychiatric symptoms of the premanifest and manifest HD participants (N=98), the results by disease stage are shown in Table 3.
| Symptom | Stage TFC | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Stage I (N=35) | Stage II (N=18) | Stage III (N=21) | Stage IV (N=21) | Stage V (N=3) | ||||||
| Present | Moderate/Severe | Present | Moderate/Severe | Present | Moderate/Severe | Present | Moderate/Severe | Present | Moderate/Severe | |
| Last Month | ||||||||||
| Apathy | 10 (28.6) | 2 (20) | 11 (61.1) | 6 (54.5) | 17 (81) | 9 (52.9) | 17 (81.0) | 16 (76.5) | 2 (66.7) | 1 (50) |
| Anxiety | 20 (57.1) | 8 (40) | 10 (55.6) | 3 (30) | 12 (57.1) | 5 (41.7) | 10 (47.6) | 4 (40) | 2 (66.7) | 1 (50) |
| Perseverative thinking/behavior | 7 (20) | 1 (14.3) | 10 (55.6) | 3 (30) | 15 (71.4) | 9 (60) | 15 (71.4) | 6 (40) | 3 (100) | 2 (66.7) |
| OCB | 4 (11.4) | 0 | 2 (11.1) | 0 | 7 (33.3) | 1 (14.3) | 6 (28.6) | 2 (33.3) | 0 | 0 |
| Depressed mood | 18 (51.4) | 3 (8.6) | 7 (38.9) | 2 (28.6) | 9 (42.9) | 1 (11.1) | 6 (28.6) | 2 (33.3) | 0 | 0 |
| Suicidal ideation | 2 (5.7) | 0 | 4 (22.2) | 1 (25) | 2 (9.5) | 1 (50) | 2 (9.5) | 1 (50) | 0 | 0 |
| Irritability | 19 (54.3) | 3 (8.6) | 8 (44.4) | 4 (50) | 10 (47.6) | 1 (20) | 14 (66.7) | 7 (50) | 1 (33.3) | 0 |
| Angry/aggressive behavior | 6 (17.1) | 2 (5.7) | 4 (22.2) | 2 (50) | 6 (28.6) | 4 (66.7) | 8 (38.1) | 6 (75) | 0 | 0 |
| Delusions/paranoid thinking | 0 | 0 | 2 (11.1) | 1 (50) | 0 | 0 | 3 (14.3) | 1 (33.3) | 1 (33.3) | 1 (100) |
| Hallucinations | 0 | 0 | 0 | 0 | 3 (14.3) | 1 (66.7) | 0 | 0 | 1 (33.3) | 0 |
| Disoriented behavior | 1 (2.9) | 0 | 2 (11.1) | 0 | 10 (47.6) | 0 | 16 (76.2) | 4 (25) | 3 (100) | 1 (33.3) |
| Last Year | ||||||||||
| Apathy | 11 (31.4) | 5 (45.5) | 11 (61.1) | 6 (54.5) | 18 (85.7) | 10 (55.6) | 17 (81) | 13 (76.5) | 2 (66.7) | 1 (50) |
| Anxiety | 22 (62.9) | 12 (54.5) | 10 (55.6) | 7 (70) | 16 (76.2) | 9 (56.3) | 10 (47.6) | 5 (50) | 2 (66.7) | 1 (50) |
| Perseverative thinking behavior | 7 (20) | 3 (42.9) | 10 (55.6) | 4 (40) | 15 (71.4) | 10 (66.7) | 15 (71.4) | 7 (46.7) | 2 (66.7) | 1 (50) |
| OCB | 3 (8.6) | 0 | 3 (16.7) | 1 (33.3) | 6 (28.6) | 1 (16.7) | 6 (28.6) | 2 (33.3) | 0 | 0 |
| Depressed mood | 19 (54.3) | 6 (31.6) | 7 (38.9) | 5 (71.4) | 10 (47.6) | 2 (20) | 6 (28.6) | 2 (33.3) | 0 | 0 |
| Suicidal ideation | 6 (17.1) | 3 (50) | 4 (22.2) | 1 (25) | 3 (14.3) | 1 (33.3) | 2 (9.5) | 1 (50) | 0 | 0 |
| Irritability | 20 (57.1) | 5 (25) | 9 (50) | 7 (77.8) | 12 (57.1) | 8 (66.7) | 14 (66.7) | 7 (50) | 1 (33.3) | 0 |
| Angry/aggressive behavior | 7 (20) | 4 (57.1) | 5 (27.8) | 2 (40) | 10 (47.6) | 8 (80) | 9 (42.9) | 8 (88.9) | 0 | 0 |
| Delusions/paranoid thinking | 1 (2.9) | 0 | 2 (11.1) | 0 | 2 (9.5) | 0 | 3 (14.3) | 1 (33.3) | 1 (33.3) | 0 |
| Hallucinations | 0 | 0 | 0 | 0 | 3 (14.3) | 2 (66.7) | 0 | 0 | 1 (33.3) | 0 |
| Disoriented behavior | 1 (2.9) | 0 | 2 (11.1) | 0 | 10 (47.6) | 16 (76.2) | 5 (31.3) | 2 (66.7) | 0 | |
a
N (% within present); HD: Huntington’s disease; OCB: obsessive-compulsive behavior.
Psychometric Properties
The internal consistency of the PBA-s, as indicated by the calculation of the Cronbach’s α coefficient, was Cronbach’s α=0.79, and most of the items contributed to it (Table 4). The independent analysis of the scores of the three raters involved in the study obtained similar results: Cronbach’s α=0.771, 0.755, and 0.602, respectively.
| Item | Symptom | Mean Value if Item Deleted | SD Value if Item Deleted | Item-Total Correlation | Cronbach’s Alpha if Item Deleted |
|---|---|---|---|---|---|
| 1 | Apathy | 6.95 | 51.066 | 0.388 | 0.781 |
| 2 | Anxiety | 6.90 | 55.162 | 0.212 | 0.799 |
| 3 | Perseveration | 7.14 | 50.050 | 0.465 | 0.772 |
| 4 | OCB | 7.91 | 58.700 | 0.129 | 0.798 |
| 5 | Depressed mood | 7.32 | 53.166 | 0.335 | 0.786 |
| 6 | Suicidal ideation | 7.90 | 47.938 | 0.685 | 0.747 |
| 7 | Irritability | 7.12 | 52.382 | 0.435 | 0.775 |
| 8 | Angry/aggressive behavior | 7.62 | 51.409 | 0.460 | 0.772 |
| 9 | Delusion | 7.90 | 46.162 | 0.663 | 0.746 |
| 10 | Hallucinations | 8.01 | 50.836 | 0.522 | 0.766 |
| 11 | Disoriented behavior | 7.63 | 48.459 | 0.579 | 0.758 |
a
OCB: obsessive-compulsive behavior.
To test the inter-rater reliability of the scale, three raters independently video recorded and evaluated 30 patients (Table 5). The test-retest reliability was also evaluated in 30 patients and was found to be very high for all items (both in severity and in frequency measurements).
| Item | Severity Scores | Frequency Scores | ||
|---|---|---|---|---|
| κ | κw | κ | κw | |
| Intra-rater | 0.87 | 1.00 | 0.87 | 0.97 |
| Inter-rater | ||||
| Pair 1 (1–2) | 0.71 | 0.92 | 0.71 | 0.91 |
| Pair 2 (1–3) | 0.64 | 0.89 | 0.70 | 0.92 |
| Pair 3 (2–3) | 0.67 | 0.90 | 0.76 | 0.93 |
| Mean | 0.67 | 0.90 | 0.72 | 0.92 |
Table 6 shows the correlations found among the items assessing similar psychopathological symptoms. All correlations found are congruent and strong enough, especially in those items measuring the same neuropsychiatric symptom, for example, irritability (r=0.918, p<0.01), hallucinations (r=0.835, p<0.01), anxiety (r=0.827, p<0.01), or apathy (r=0.828, p<0.01). Other items, such as angry/aggressive behavior, correlate strongly with agitation (r=0.829, p<0.01), as well as depressed mood and dysphoria (r=0.835, p<0.01), as expected.
| PBA-s | NPI | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Delusion | Hallucination | Agitation | Dysphoria | Anxiety | Euphoria | Apathy | Disinhibition | Irritability | AMB | |
| Apathy | 0.182 | 0.152 | 0.008 | –0.034 | –0.042 | –0.188 | 0.828** | 0.032 | –0.011 | –0.066 |
| Anxiety | 0.128 | 0.292* | 0.278 | 0.338* | 0.827** | –0.139 | 0.200 | –0.079 | 0.399** | –0.138 |
| Perseveration | 0.233 | –0.029 | 0.311* | –0.007 | 0.215 | 0.116 | 0.291* | 0.181 | 0.340* | 0.098 |
| OCB | 0.197 | 0.326* | 0.141 | 0.006 | 0.209 | 0.383** | 0.085 | 0.392** | 0.491** | 0.295* |
| Depressed mood | –0.226 | –0.171 | 0.331* | 0.835** | 0.236 | –0.097 | –0.063 | –0.171 | 0.175 | –0.200 |
| Suicidal ideation | 0.034 | 0.132 | 0.325* | 0.427** | 0.242 | –0.063 | 0.002 | 0.132 | 0.421** | –0.130 |
| Irritability | 0.105 | 0.164 | 0.576** | 0.042 | 0.291* | 0.065 | 0.082 | 0.192 | 0.918** | 0.039 |
| Angry/aggressive behavior | 0.068 | –0.155 | 0.829** | 0.232 | 0.269 | –0.087 | 0.016 | 0.026 | 0.592** | –0.020 |
| Delusion | 0.599** | –0.111 | 0.188 | –0.076 | 0.028 | –0.063 | 0.169 | –0.111 | 0.123 | 0.073 |
| Hallucination | 0.140 | 0.835** | –0.024 | –0.190 | 0.064 | –0.046 | 0.153 | 0.251 | 0.212 | –0.095 |
| Disoriented behavior | 0.139 | 0.013 | 0.288 | –0.237 | 0.087 | 0.012 | 0.499** | 0.043 | 0.227 | 0.057 |
a
AMB: Aberrant motor behavior; NPI: Neuropsychiatric Inventory; OCB: obsessive-compulsive behavior.
*
Correlation is significant at the 0.05 level (p<0.05; two-tailed); ** correlation is significant at the 0.01 level (p<0.01; two-tailed).
Stage TFC was significantly correlated with five PBA-s items: apathy (r=0.564, p<0.001), perseveration (r=0.525, p<0.001), delusion/paranoid thinking (r=0.340, p<0.001), hallucinations (r=0.266, p<0.001), and disorientation (r=0.724, p<0.001), while not the rest: anxiety (r=−0.101, p=0.280), obsession (r=0.182, p=0.051), depression (r=−0.153, p=0.102), suicide (r=0.088, p=0.347), irritability (r=0.050, p=0.597), and aggressiveness (r=0.177, p=0.058).
Factor Analysis
After running the factanal function with an increasing number of factors, the factor analysis solution with four factors was the first solution to adequately explain the data (χ2=8.87, df=6, p=0.181), accounting for 56% of the total variance. As summarized in Table 7, the four latent factors found may be related to irritability, affective alterations, apathy, and perseverance.
| Factor 1 | Factor 2 | Factor 3 | Factor 4 |
|---|---|---|---|
| Irritability 18% of Variance | Depression 15% of Variance | Apathy 13% of Variance | Perseverance 10% of Variance |
| Aggressiveness (0.98) | Suicidal ideation (0.97) | Apathy (0.97) | Perseverance (0.74) |
| Irritability (0.54) | Depressed mood (0.46) | Obsessive (0.46) |
a
For each one of the four factors, the amount of explained variance is reported together with symptoms associated to the factor (loadings >0.4); factor loadings for listed symptoms are reported in parentheses.
Discussion
To our knowledge, this is the first time that the PBA-s scale has been validated in another language. The results suggest that the Spanish version of the PBA-s is a valid and reliable scale for measuring the severity and frequency of neuropsychiatric symptoms in HD patients.
Regarding inter-rater reliability, our results are similar to those Callaghan et al.14 obtained in their recent study measuring the psychometric properties of the PBA-s (κw were 0.94 for severity scores and 0.92 for frequency scores in a sample of 410 individuals). Although using the PBA-HD, we can also affirm that our results are in line with those of Craufurd et al.5 (κw were 0.86 for severity scores and 0.84 for frequency scores in a sample of 38 individuals included in the reliability analysis) and Kingma et al.13 (κw=0.86 for severity scores and 0.84 for frequency scores in a subset of 63 subjects). Test-retest reliability also achieved substantial agreement, in line with that observed in the original Craufurd et al. study5 (κw==0.94 for severity scores and 0.70 for frequency scores in a subset of 15 subjects). This psychometric property was not reflected in the studies by Callaghan et al. or Kingma et al.
The internal consistency was good (α=0.79) and was equivalent to the results in Kingma et al.’s study.13 The PBA-s correlates strongly with the NPI, a validated scale frequently used to assess psychopathology in Alzheimer’s disease and other brain disorders with a similar structure, showing its concurrent validity and therefore its accuracy in assessing neuropsychiatric symptoms in HD patients.
As found in other studies, apathy is strongly correlated with TFC stages.5,13,23 Correlation with TFC was also found to be significant in these symptoms: perseveration, delusion/paranoid thinking, hallucinations, and disorientation, showing their increasing prevalence in more advanced stages of HD, although the low prevalence of the psychotic symptoms should be considered. The remaining symptoms do not correlate, perhaps due to the different prevalence and severity scores encountered within the disease’s stages. Although not significantly, anxiety and depression were found to correlate inversely with TFC stages, according to other studies.24
The results of the factor analysis indicate four latent factors, accounting for 56% of the total variance—apathy, irritability, depression, and perseveration—which is consistent with similar analyses performed by other authors. Callaghan et al.14 extracted three factors after a principal-components analysis: apathy, irritability, and depression. The same factors were identified in studies conducted by Craufurd et al.5 and Kingma et al.13, using the PBA-HD. Rickards et al.25 performed a factor analysis on 1,690 completed UHDRS-behavioral assessments, finding four factors: depression, drive/executive function, irritability/aggression, and psychosis. In their study, the factor “drive/executive function” included apathy, perseveration, and compulsion. In our study, “perseveration” includes perseverative thinking/behavior and obsessive-compulsive thinking/behavior. This fact might show a behavioral cluster involving the frontal lobe dysfunction.
Our results showed that neuropsychiatric symptoms are very frequent in HD. In our sample, only eight participants (6.8%) scored 0 on all of the items during last month, a much smaller percentage than that encountered in other studies. Van Dujin et al.,8 using the UHDRS-b, found that the 27% of the participants had a score of 0 on all five behavioral subscales in the previous month. This may be explained by taking into account the profile of our participants, most of whom were assessed in a neuropsychiatry unit, where they were provided care specifically due to the presence of psychiatric symptomatology. Focusing on manifest HD patients, apathy was the most frequent symptom during the previous month, as found in other studies.5 Anxiety was the second most frequent, followed by irritability and disoriented behavior. Apathy, perseverative thinking/behavior, and angry/aggressive behavior correlated with TFC in the evaluation of the previous year. Psychotic symptoms, hallucinations, and delusions were less frequent, showing that psychosis is a very infrequent finding in this population, as encountered in other studies.7 Regarding the presence of neuropsychiatric symptomatology, depending on functional stage, our results, despite being evaluated by different scales, are quite similar to those Van Dujin et al.8 obtained, although they made their analysis based on clusters that were previously reported in another study after a principal-components analysis.6
Although we cannot make an exact comparison with Callaghan et al.’s study results,14 our subset of 98 mutation carriers presents higher percentages of neuropsychiatric symptoms, especially moderate–severe symptoms (scores 3 and 4).
A very important symptom, suicidal ideation, is present in 11.2% of the mutation carrier group, similar to that encountered by Callaghan et al.14 (11.7%) and higher than that in Craufurd et al.’s5 study using the PBA-HD (8.0%) and in Hubers et al.’s26 study (8.0%) using the UHDRS-b. If only cases with moderate-severe scores are considered, the prevalence decreases to 3.3%. Nevertheless, this figure is higher than the 1.5% figure Callaghan et al.14 reported.
It is important to note that in the control group without HD family history (caregivers), there is a high prevalence of anxiety, irritability, and depressed mood, revealing the impact of the HD care provision on the mental health of caregivers. This is an issue that is very important to note in the comprehensive treatment of the disease, which should include the whole family afflicted by HD.27
This study has several noteworthy strengths. It followed a strict methodology for the cultural adaptation and validation of the scale, systematically analyzing the main psychometric characteristics. This is a multicenter study, performed by different evaluators with different types of clinical training (psychologists, psychiatrists, neurologists, nurses). The study was carried out using a large sample, with 98 patients with HD and 19 healthy controls and relatives, in different clinical settings (inpatients and outpatients). Unlike previous studies, our study aimed to highlight the severity of neuropsychiatric symptoms, showing their prevalence in a stratified manner that distinguishes mild severity (scores 1, 2) from moderate to severe (scores 3, 4) for each of the different categories of participants. Severity of symptoms has also been presented both in the last month and in the last year, according to the original structure of the PBA-s, something that had not previously been done.
There also are several limitations to this work that merit mention. Having the greatest number of assessments carried out in the neuropsychiatry unit of the Hospital Mare de Déu de la Mercè may have biased the results, as this unit focuses on the treatment of neuropsychiatric symptoms in patients with HD. The samples used for the analysis of reliability may not be great in terms of number, but ultimately, 90 assessments conducted on 30 subjects were available. A further limitation of our study is that we could only use NPI with 46 patients because this instrument is intended to be completed by a well-informed caregiver, and in some cases, no such figure was available.
In our experience, training is fundamental to increase the reliability of the scale. The authors recommend using suggested prompts in a nonrestrictive manner and changing the order of the items if necessary. We decided to commence the assessment focusing on the less distressing items to ameliorate anxiety due to examination and to improve patient collaboration. Moreover, we recommend using the suggested prompts more literally, thereby ensuring greater reliability.
Conclusions
This study provides evidence of the validity of the Spanish version of the PBA-s, showing good internal validity, intra-rater and inter-rater reliability, and strong correlations with other psychopathological rating scales. A final version is now available for use by Spanish language professionals.
Acknowledgments
The authors thank all those who took part in this study, as well as their paid and family caregivers for participating in the study.
Footnotes
The funding organization played no role in the study design, data collection and analysis, or manuscript approval.
The Spanish Huntington Disease Network for the present study are: José M. Arbelo (Movement Disorders Unit, Department of Neurology, Hospital Insular de Gran Canaria, Las Palmas de Gran Canaria, Spain), Koldo Berganzo (Department of Neurology, Hospital de Cruces, Barakaldo, Vizcaya, Spain), Maria C. Durán-Herrera (Hospital Infanta Cristina, Badajoz, Spain), José M. García-Moreno (Department of Neurology, Hospital Universitario Virgen de la Macarena, Sevilla, Spain), Patrocinio García-Moreno (Hospital Infanta Cristina, Badajoz, Spain), Rocío García-Ramos (Hospital Clínico, Madrid, Spain), Juan C. Gómez-Esteban (Department of Neurology, Hospital de Cruces, Barakaldo, Vizcaya, Spain), Jaume Kulisevsky (Hospital de la Santa Creu i Sant Pau, Barcelona, Spain; CIBERNED, Spain), Inés Legarda (Hospital Son Espases, Palma de Mallorca, Spain), María del Valle Loarte (Hospital de Fuenlabrada, Madrid, Spain), Javier López del Val (Hospital Clínico Universitario Lozano Blesa, Zaragoza, Spain), Asunción Martinez-Descals (Language Area Coordinator, European Huntington Disease Network), Pablo Mir (Unidad de Trastornos del Movimiento, Servicio de Neurología y Neurofisiología Clínica, Instituto de Biomedicina de Sevilla, Hospital Universitario Virgen del Rocío/CSIC/Universidad de Sevilla, Seville, Spain; CIBERNED, Spain), Blas Morales (Hospital Universitario San Cecilio, Granada, Spain), María A. Ramos-Arroyo (Hospital Virgen del Camino, Pamplona, Spain), René Ribacoba (Department of Neurology, Hospital Universitario Central Asturias and Hospital Alvarez Buylla-Mieres, Asturias, Spain), Maria A. Zea (Fundación CIEN Instituto de Salud Carlos III, Madrid, Spain).
References
1.
Shoulson I, Young AB: Milestones in Huntington disease. Mov Disord 2011; 26:1127–1133
2.
Biglan KM, Zhang Y, Long JD, et al: Refining the diagnosis of Huntington disease: the PREDICT-HD study. Front Aging Neurosci 2013; 5:12
3.
Duff K, Paulsen JS, Beglinger LJ, et al: Psychiatric symptoms in Huntington’s disease before diagnosis: the predict-HD study. Biol Psychiatry 2007; 62:1341–1346
4.
Stout JC, Paulsen JS, Queller S, et al: Neurocognitive signs in prodromal Huntington disease. Neuropsychology 2011; 25:1–14
5.
Craufurd D, Thompson JC, Snowden JS: Behavioral changes in Huntington disease. Neuropsychiatry Neuropsychol Behav Neurol 2001; 14:219–226
6.
Reedeker W, van der Mast RC, Giltay EJ, et al: Psychiatric disorders in Huntington’s disease: a 2-year follow-up study. Psychosomatics 2012; 53:220–229
7.
Thompson JC, Harris J, Sollom AC, et al: Longitudinal evaluation of neuropsychiatric symptoms in Huntington’s disease. J Neuropsychiatry Clin Neurosci 2012; 24:53–60
8.
van Duijn E, Craufurd D, Hubers AA, et al: Neuropsychiatric symptoms in a European Huntington’s disease cohort (REGISTRY). J Neurol Neurosurg Psychiatry 2014; 85:1411–1418
9.
Huntington Study Group: Unified Huntington’s Disease Rating Scale: reliability and consistency. Mov Disord 1996; 11:136–142
10.
Aubeeluck AV, Buchanan H, Stupple EJ: “All the burden on all the carers”: exploring quality of life with family caregivers of Huntington’s disease patients. Qual Life Res 2012; 21:1425–1435
11.
Hocaoglu MB, Gaffan EA, Ho AK: Health-related quality of life in Huntington’s disease patients: a comparison of proxy assessment and patient self-rating using the disease-specific Huntington’s disease health-related quality of life questionnaire (HDQoL). J Neurol 2012; 259:1793–1800
12.
Orth M, Handley OJ, Schwenke C, et al: Observing Huntington’s disease: the European Huntington’s Disease Network’s REGISTRY. PLoS Curr 2010; 2:RRN1184
13.
Kingma EM, van Duijn E, Timman R, et al: Behavioural problems in Huntington’s disease using the Problem Behaviours Assessment. Gen Hosp Psychiatry 2008; 30:155–161
14.
Callaghan J, Stopford C, Arran N, et al: Reliability and factor structure of the Short Problem Behaviors Assessment for Huntington’s disease (PBA-s) in the TRACK-HD and REGISTRY studies. J Neuropsychiatry Clin Neurosci 2015; 27:59–64
15.
McNally G, Rickards H, Horton M, et al: Exploring the validity of the short version of the Problem Behaviours Assessment (PBA-s) for Huntington’s disease: a Rasch analysis. J Huntingtons Dis 2015; 4:347–369
16.
Clinicaltrials.gov: Identifier: NCT01574053. https://clinicaltrials.gov/.
17.
Cummings JL: The Neuropsychiatric Inventory: assessing psychopathology in dementia patients. Neurology 1997; 48(Suppl 6):S10–S16
18.
Sousa VD, Rojjanasrirat W: Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. J Eval Clin Pract 2011; 17:268–274
19.
Cronbach LJ, Warrington WG: Time-limit tests: estimating their reliability and degree of speeding. Psychometrika 1951; 16:167–188
20.
Cohen J: A coefficient of agreement for nominal scales. Educ Psychol Meas 1960; 20:37–46
21.
Cohen J: Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70:213–220
22.
Team RC: R: A language and environment for statistical computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2013. Available internet at http://www.r-project.org. Accessed 2015.
23.
Hamilton JM, Salmon DP, Corey-Bloom J, et al: Behavioural abnormalities contribute to functional decline in Huntington’s disease. J Neurol Neurosurg Psychiatry 2003; 74:120–122
24.
Paulsen JS, Nehl C, Hoth KF, et al: Depression and stages of Huntington’s disease. J Neuropsychiatry Clin Neurosci 2005; 17:496–502
25.
Rickards H, De Souza J, van Walsem M, et al: Factor analysis of behavioural symptoms in Huntington’s disease. J Neurol Neurosurg Psychiatry 2011; 82:411–412
26.
Hubers AA, van Duijn E, Roos RA, et al: Suicidal ideation in a European Huntington’s disease population. J Affect Disord 2013; 151:248–258
27.
Simpson SA, Rae D: A standard of care for Huntington’s disease: who, what and why. Neurodegener Dis Manag 2012; 2:1–5
Information & Authors
Information
Published In
History
Received: 12 February 2016
Revision received: 22 April 2016
Accepted: 23 April 2016
Published online: 15 July 2016
Published in print: Winter 2017
Authors
Competing Interests
The authors report no financial relationships with commercial interests.
Funding Information
Agència de Gestió d\'Ajuts Universitaris i de Recerca: 2014SGR1573
Supported by the Catalonian Government (2014SGR1573 to the Research Unit of FIDMAG Hermanas Hospitalarias).Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).