The PBA-HD is a semistructured behavioral interview designed for rating the presence, frequency, and severity of behavioral abnormalities in patients with Huntington’s disease.
5 The PBA-HD is a 40-item instrument intended for use by trained psychiatrists with experience in the assessment of patients with neuropsychiatric disorders. The scale was developed via the compilation of psychiatric symptoms that patients and relatives frequently reported or those that are often referred to in the literature due to clinical interest in the field of dementia syndromes affecting frontal lobes.
5 The instrument was developed in two phases. First, a five-point (0–4) scale was included for rating the severity and frequency of each symptom recorded as present or absent for the period since the patient’s last visit to the clinic, as done in the UHDRS. Second, strict scoring criteria for rating the severity of each symptom were included, and frequency criteria for every symptom were also defined to improve reliability. In order to maintain compatibility with the UHDRS Behavioral Scale, the 4-week period immediately prior to the interview was chosen for the assessment. Relative to its psychometric properties, the PBA-HD showed good feasibility; in their original study, the authors referred to good reliability (inter-rater mean weighted kappa: 0.86; test-retest mean weighted kappa: 0.94).
5 Kingma et al.
13 created a Dutch translation of the PBA-HD in 2008. This version was able to replicate the previous results: an interrater reliability of 0.82 for severity scores and 0.73 for frequency scores in a larger sample than that used in Craufurd et al.’s original study.
The PBA-s,
12 an 11-item semistructured clinical interview, was developed by the EHDN Behavioral Phenotype Working Group for use in the REGISTRY study,
12 where a shorter scale was necessary. The PBA-s is an instrument that evaluates severity and frequency of these symptoms: depressed mood, suicidal ideation, anxiety, irritability, angry/aggressive behavior, apathy, perseveration, obsessive-compulsive behavior, paranoid thinking, hallucinations, and disoriented behavior. To improve reliability, suggested prompts are offered to the interviewers, and criteria for rating frequency and severity are provided. The frequency scores range from 0
(symptom absent) to 4
(present all the time). The severity scores are defined independently for each item, ranging from 0
(symptom absent) to 4
(symptom causing severe problems). Severity and frequency scores are multiplied to obtain a total score for each item. The final score is rated considering all available information: that given by the patient and relatives, as well as the interviewer’s own observations of the patient’s behavior. Patients and informants are asked to identify the presence or absence of the symptoms during the last month, but in order not to exclude important symptoms that could have occurred during the last year (the recommended period to apply the scale within the longitudinal studies that are usually performed in the HD population), a “worst” score is added to set the severity of the symptom during that period of time.
Methods
Design and Participants
A prospective observational multicenter study was carried out between January 2013 and December 2014. One hundred seventeen participants were recruited with the same criteria from five Spanish centers within the European Huntington Disease Network: 80 (68.4%) from the Hospital Mare de Déu de la Mercè (Barcelona), 18 (15.38%) from the Hospital Ramón y Cajal (Madrid), 11 (9.4%) from the Hospital Universitario (Burgos), five (4.27%) from the Hospital Clinic (Barcelona), and three (2.56%) from the Hospital Virgen de Arriaxaca (Murcia). The Ethics Committee, CEIC Hermanas Hospitalarias (Barcelona, Spain), approved the study protocol in accordance with the Declaration of Helsinki of 1964, revised in Edinburgh in 2000. Written informed consent of the participants was obtained.
The group of participants was comprised of 117 subjects (104 outpatients and 13 inpatients). The distribution within the inpatient group was nine patients admitted to the neuropsychiatry hospitalization ward and four patients admitted to the neuropsychiatry day care hospital. The mean stay of these inpatients was 148 days. Patients were categorized into five groups: people at risk for HD, premanifest expansion mutation carriers, manifest HD patients, control expansion negative, and control without a family history of HD, following the structure of the REGISTRY study. In our sample, 77 participants were manifest HD patients, nine people were at risk, 12 were asymptomatic carriers, and 19 were controls. Informants were coded following the instructions of the author as follows: spouse or partner (N=34), parent (N=12), sibling (N=10), child (N=5), other relative (N=2), friend or neighbor (N=1), professional care worker (N=10), other (N=3), no informant-subject came alone (N=40).
Measures
A sociodemographic data sheet and three questionnaires were used: PBA-s, UHDRS Total Functional Capacity (TFC), and Neuropsychiatric Inventory (NPI).
17 The NPI was administered the same day but after the PBA-s, with the informant alone; thus, this instrument is based on responses given by a well-informed caregiver. The sociodemographic data sheet provided information about many characteristics of the group, including age, gender, educational level, and other variables, such as CAG repetition number when suitable, disease stage, and rater category.
The TFC is an ordinal component of the UHDRS. Scores on the TFC represent five stages in the neurodegenerative disease process. Lower scores represent greater functional impairment: stage I represents scores of 13–11; stage II, scores of 10–7; stage III, scores of 6–3; stage IV, scores of 2–1; and stage V, a score of 0.
The NPI is an instrument designed to capture most of the more frequent psychiatric and behavioral symptoms in dementia, with a similar structure to the PBA-s. Twelve items are included: delusions, hallucinations, agitation, dysphoria, anxiety, apathy, irritability, euphoria, disinhibition, aberrant motor behavior, nighttime behavior disturbances, and appetite and eating abnormalities. The frequency and severity of every item are measured on the basis of scripted questions administered to the patient’s caregiver, and a total score of every single item is obtained by multiplying the scores of the different domains.
Procedure
The validation process was carried out following the recommended methodological approaches for translation, adaptation, and cross-cultural validation of research instruments.
18 The process began by first making a direct translation, which was done by professionals with a suitable level of English and psychopathological expertise and who were experienced in Huntington’s disease. Each translator performed his or her translation independently, and, after several meetings, a consensus version was obtained. Psychiatrists and neuropsychologists from other units of the hospital, unrelated to HD, reviewed the first version in order to evaluate its comprehensibility and suitability. A pilot study was carried out using a limited number of subjects, and this was designed to evaluate the instrument’s feasibility (evaluate the instructions, items, and response format clarity as well as application time). Afterward, and once this preliminary version was reviewed, a bilingual translator created a back translation to compare the differences between the translated version and the original text.
To evaluate reliability, we videotaped the interview of the first 30 patients enrolled consecutively and who gave informed consent to perform it. One rater (JR-I) re-evaluated his own interview after one month to measure intra-rater reliability, and two different raters (MF and CM) evaluated their interviews independently to evaluate inter-rater reliability. Concurrent validity was also measured. We chose to compare the results of the PBA-s Spanish version with those obtained by simultaneously applying the NPI.
Statistical Analysis
Descriptive statistics of the reference group are reported. Internal consistency was tested using the Cronbach’s α coefficient.
19 Intra- and inter-rater reliability were assessed using Cohen’s kappa (κ).
20 Cohen’s kappa takes into account disagreement between the two raters, but not the degree of disagreement. This is especially relevant when the ratings are ordered, and therefore, the weighted kappa (κ
w) was also calculated.
21 Weighted kappa allows scores to differ by 1 point. A Spearman correlation coefficient was computed between the total scores of the PBA-s and NPI (N=46), to test the concurrent validity of the scale, and between the PBA-s and the TFC scores. This statistical analysis was performed using the IBM SPSS Statistics (version 23.0).
In order to evaluate the factor structure of the severity of symptoms, we carried out a factor analysis on the severity scores (
Table 1). Specifically, we applied the factanal function from the R statistical package
22 to perform a maximum-likelihood factor analysis with varimax rotation. To avoid confounding effects from both unrelated controls and former patients with negative test outcomes, these two subgroups were excluded from the analysis (leaving an N=98). From the pool of symptoms, “delusions” and “hallucinations” were discarded, as less than 10% of individuals scored nonnull values for the severity of both symptoms.
Discussion
To our knowledge, this is the first time that the PBA-s scale has been validated in another language. The results suggest that the Spanish version of the PBA-s is a valid and reliable scale for measuring the severity and frequency of neuropsychiatric symptoms in HD patients.
Regarding inter-rater reliability, our results are similar to those Callaghan et al.
14 obtained in their recent study measuring the psychometric properties of the PBA-s (κ
w were 0.94 for severity scores and 0.92 for frequency scores in a sample of 410 individuals). Although using the PBA-HD, we can also affirm that our results are in line with those of Craufurd et al.
5 (κ
w were 0.86 for severity scores and 0.84 for frequency scores in a sample of 38 individuals included in the reliability analysis) and Kingma et al.
13 (κ
w=0.86 for severity scores and 0.84 for frequency scores in a subset of 63 subjects). Test-retest reliability also achieved substantial agreement, in line with that observed in the original Craufurd et al. study
5 (κ
w==0.94 for severity scores and 0.70 for frequency scores in a subset of 15 subjects). This psychometric property was not reflected in the studies by Callaghan et al. or Kingma et al.
The internal consistency was good (α=0.79) and was equivalent to the results in Kingma et al.’s study.
13 The PBA-s correlates strongly with the NPI, a validated scale frequently used to assess psychopathology in Alzheimer’s disease and other brain disorders with a similar structure, showing its concurrent validity and therefore its accuracy in assessing neuropsychiatric symptoms in HD patients.
As found in other studies, apathy is strongly correlated with TFC stages.
5,13,23 Correlation with TFC was also found to be significant in these symptoms: perseveration, delusion/paranoid thinking, hallucinations, and disorientation, showing their increasing prevalence in more advanced stages of HD, although the low prevalence of the psychotic symptoms should be considered. The remaining symptoms do not correlate, perhaps due to the different prevalence and severity scores encountered within the disease’s stages. Although not significantly, anxiety and depression were found to correlate inversely with TFC stages, according to other studies.
24The results of the factor analysis indicate four latent factors, accounting for 56% of the total variance—apathy, irritability, depression, and perseveration—which is consistent with similar analyses performed by other authors. Callaghan et al.
14 extracted three factors after a principal-components analysis: apathy, irritability, and depression. The same factors were identified in studies conducted by Craufurd et al.
5 and Kingma et al.
13, using the PBA-HD. Rickards et al.
25 performed a factor analysis on 1,690 completed UHDRS-behavioral assessments, finding four factors: depression, drive/executive function, irritability/aggression, and psychosis. In their study, the factor “drive/executive function” included apathy, perseveration, and compulsion. In our study, “perseveration” includes perseverative thinking/behavior and obsessive-compulsive thinking/behavior. This fact might show a behavioral cluster involving the frontal lobe dysfunction.
Our results showed that neuropsychiatric symptoms are very frequent in HD. In our sample, only eight participants (6.8%) scored 0 on all of the items during last month, a much smaller percentage than that encountered in other studies. Van Dujin et al.,
8 using the UHDRS-b, found that the 27% of the participants had a score of 0 on all five behavioral subscales in the previous month. This may be explained by taking into account the profile of our participants, most of whom were assessed in a neuropsychiatry unit, where they were provided care specifically due to the presence of psychiatric symptomatology. Focusing on manifest HD patients, apathy was the most frequent symptom during the previous month, as found in other studies.
5 Anxiety was the second most frequent, followed by irritability and disoriented behavior. Apathy, perseverative thinking/behavior, and angry/aggressive behavior correlated with TFC in the evaluation of the previous year. Psychotic symptoms, hallucinations, and delusions were less frequent, showing that psychosis is a very infrequent finding in this population, as encountered in other studies.
7 Regarding the presence of neuropsychiatric symptomatology, depending on functional stage, our results, despite being evaluated by different scales, are quite similar to those Van Dujin et al.
8 obtained, although they made their analysis based on clusters that were previously reported in another study after a principal-components analysis.
6Although we cannot make an exact comparison with Callaghan et al.’s study results,
14 our subset of 98 mutation carriers presents higher percentages of neuropsychiatric symptoms, especially moderate–severe symptoms (scores 3 and 4).
A very important symptom, suicidal ideation, is present in 11.2% of the mutation carrier group, similar to that encountered by Callaghan et al.
14 (11.7%) and higher than that in Craufurd et al.’s
5 study using the PBA-HD (8.0%) and in Hubers et al.’s
26 study (8.0%) using the UHDRS-b. If only cases with moderate-severe scores are considered, the prevalence decreases to 3.3%. Nevertheless, this figure is higher than the 1.5% figure Callaghan et al.
14 reported.
It is important to note that in the control group without HD family history (caregivers), there is a high prevalence of anxiety, irritability, and depressed mood, revealing the impact of the HD care provision on the mental health of caregivers. This is an issue that is very important to note in the comprehensive treatment of the disease, which should include the whole family afflicted by HD.
27This study has several noteworthy strengths. It followed a strict methodology for the cultural adaptation and validation of the scale, systematically analyzing the main psychometric characteristics. This is a multicenter study, performed by different evaluators with different types of clinical training (psychologists, psychiatrists, neurologists, nurses). The study was carried out using a large sample, with 98 patients with HD and 19 healthy controls and relatives, in different clinical settings (inpatients and outpatients). Unlike previous studies, our study aimed to highlight the severity of neuropsychiatric symptoms, showing their prevalence in a stratified manner that distinguishes mild severity (scores 1, 2) from moderate to severe (scores 3, 4) for each of the different categories of participants. Severity of symptoms has also been presented both in the last month and in the last year, according to the original structure of the PBA-s, something that had not previously been done.
There also are several limitations to this work that merit mention. Having the greatest number of assessments carried out in the neuropsychiatry unit of the Hospital Mare de Déu de la Mercè may have biased the results, as this unit focuses on the treatment of neuropsychiatric symptoms in patients with HD. The samples used for the analysis of reliability may not be great in terms of number, but ultimately, 90 assessments conducted on 30 subjects were available. A further limitation of our study is that we could only use NPI with 46 patients because this instrument is intended to be completed by a well-informed caregiver, and in some cases, no such figure was available.
In our experience, training is fundamental to increase the reliability of the scale. The authors recommend using suggested prompts in a nonrestrictive manner and changing the order of the items if necessary. We decided to commence the assessment focusing on the less distressing items to ameliorate anxiety due to examination and to improve patient collaboration. Moreover, we recommend using the suggested prompts more literally, thereby ensuring greater reliability.