1. Naturalistic Reasoning and Problem-Solving (PRO Code)
Deficits in executive functioning, or reasoning and problem solving, are some of the most consistently identified cognitive problems in schizophrenia.
98,99 Also, individuals with schizophrenia demonstrate deficits in social problem-solving, including problem-recognition, problem-definition, and generating alternatives.
100–102 Naturalistic instances of everyday problem-solving are common, often verbalized, and observable on videotape.
For guidance to define a category of
problem-solving as performed in-situ, the team looked to neuropsychological test paradigms. In these paradigms, reasoning and problem-solving is a process by which problems are identified and solved by abstract reasoning.
103 Test paradigms for this domain assess a variety of skills, including spatial skills,
104,105 set-shifting, response initiation and suppression, and nonverbal components rule selection.
106 We focused on the Wisconsin Card-Sort Test (WCST), the Rey-Osterrieth Complex Figure Test (RCFT),
107 the Tower of London,
108 the Independent Living Scales–Problem-Solving Factor subscale (ILS–PB),
109 and the Neuropsychological Assessment Battery–Mazes (NAB–Mazes). The Tower of London and the RCFT involve manipulating unusual stimuli to achieve a highly structured goal, making these tests difficult to compare with real-world problems.
110 In the video, we could find corollaries to the set-shifting skills tested by the WCST, to the time-limited problem-solving required for the NAB–MAZES, and the skills required to identify a solution to a hypothetical problem as posed in the ILS–PB. These tests require the subject to recognize that a novel circumstance presents itself, to respond effectively to that circumstance, to generate solutions in a timely manner, and/or to weigh a range of potential solutions.
Social problem-solving assessments and problem-solving therapies emphasize similar skills. Problem-solving therapies stress the identification of problems, positive problem-solving orientation, solution-generation, and effective response. Social problem-solving assessments test skills in identifying problems (called receiving), generating solutions (processing), and choosing effective solutions (sending). To map these skills to video data, we looked for reliable indicators that the subject faced a challenge (e.g., when a subject did not know how to turn on the oven). However, mere identification of a problem (“I need some cigarettes;” “I need a bigger apartment.”) could not be coded, because, in some of these instances, problem-solving could reasonably be postponed or might never be pursued. We code problem-solving behavior only when the identification of a problem is followed by action.
Our first few subjects demonstrated an aptitude for problem-identification and solution-generation in the course of efforts to help others. One subject repeatedly clarified what the ethnographer would need to do to take the bus back to his car. He explained how much change was needed, pointed out the corner at which to wait, estimated the wait-time, advised the ethnographer how to know when the bus approached the appropriate street, and so on. On other occasions, he provided information to orient the ethnographer (“I'm going to change my shirt, and then we can go.”). The subject was anticipating hypothetical problems and aiming to prevent them.
Then, we recognized many anticipatory behaviors that did more than fix a present challenge. Subjects converted a situation that was not problematic per se into an opportunity to improve the present. These behaviors involved generating options. One subject took a seat on a bus bench as she and the ethnographer walked past. “I'm not taking the bus; I'm just going to rest here,” she said; she was tired, and she took advantage of the bench to address that. Another subject looked at his watch as he took the bus to class; “we will have time to have another cigarette before class,” he said. Another subject, recalling the need to go downstairs, pointed to the elevator and asked, “We could take the elevator, or we could take the stairs. Which should we take?” Both anticipating potential problems and identifying opportunities require the same planning and reasoning skills as problem-solving.
After review of these examples and successive trials, we defined the problem-solving code: We mark problem-solving when we see problem-identification, positive problem-orientation, and solution-generation together. We include “helping behaviors” and explanations for others when they are both a response to an explicit need and when they have implications for action in the moment. In other words, problem-solving (PRO) behaviors are those in which subjects are seen to manage a novel circumstance, satisfy a need, or fix complications in the present. PRO behaviors may also be explicit, in-the-moment attempts to improve a situation. PRO behaviors include planning whether there is time to stop at the store and telling another person how to get to a landmark when such a need is expressed. In pilot data, raters agreed 78.3% (standard error [SE]: 0.09) of the time that problem-solving was present or absent. Across all nine subjects, PRO behaviors occurred approximately once per hour. However, problems could be identified and solved in successive stages over a considerable period of time (such as when a subject said, “I need to get a ride from my sister,” and made three phone calls to three different people over the course of an hour). The time required to solve a problem was not rated. Instead, we rated the kinds of skills used to solve each problem. Problem-Solving Skill, a 3-point scale, rewards anticipatory and creative—as opposed to rote and rudimentary—problem-solving. Since, in our pilot work, problem-solving instances occurred approximately once per hour, we could achieve adequate sampling in an observational period of 10–15 hours. The interrater reliability for PRO skill rating was excellent (kappa: 0.87; SE: 0.05), and the internal-consistency reliability, or day-to-day stability of PRO skill use (i.e., from one session to the next), was good (Cronbach’s alpha: 0.68).
2. Social Interactions (INT Code)
Social cognitive deficits and impairments in social functioning are high-priority areas in schizophrenia research.
111,112 Social cognitive deficits may mediate the relationship between neurocognition and functioning,
2 and social functioning itself is an important outcome.
113 An NIMH workshop defined social cognition in schizophrenia as comprising five skill areas: theory of mind (or mental state attribution), social perception, social knowledge, attributional bias, and emotional processing.
114 In general, these processes are too discrete or implicit to be identified in videos. Statements of attributional bias can be coded in transcripts,
115 but we did not have structured narratives for comparison across subjects.
Pilot subjects spent a great deal of time reporting social behaviors (e.g., “I'm going to class tomorrow with my brother.”). One subject described friends down the hall, a boyfriend, and a granddaughter. Yet subjects' reports of interactions did not always correspond to social activity. One subject mentioned frequently talking to God, and another talked intimately about Oprah Winfrey; a third spoke aloud to someone else when the ethnographer left the room; and many of the individuals mentioned by subjects were never seen to visit or call. We found these reports of social interactions difficult to evaluate. If we could not confirm that the subject really spent the afternoon with a brother, as reported, we could not justify coding this information as an indicator of social activity. And, although we found it significant that some subjects reported receiving substantial psychosocial support from relationships that were best understood as intra- rather than inter- personal (e.g., Oprah Winfrey), we felt we could not code these as social ties. Ultimately, we instead chose to design a coding strategy to take advantage of video ethnography’s unique ability to document manifest social interactions. Video can document the interactions with others that actually take place in the course of subjects' everyday routines, an aspect of subjects' social lives that is very rarely studied.
In our coding, social interactions are timed from beginning to end. They begin when the subject engages in an interaction and end when the interlocutors part; the joining in of a third individual is counted as a new interaction. Interactions that take place over the phone, by text message, or e-mail are rated identically as in-person interactions. Tracking duration allows for calculation of the average length of interactions, the percent of interactions that are short (less than 30 seconds) or long (e.g., longer than 3 minutes), and the percent of a subject’s total time spent in interaction. Interaction duration measurements reached very high interrater agreement (kappa: 0.99), but stability of mean interaction duration from session to session was not high; α reliability improved to 0.51 when Sessions 1 and 2 were compared with Sessions 3 and 4, suggesting that longer periods of observation may be required to assess average interaction length.
Also, we rated the quality of the interaction with a scale of
relatedness. We adapted the Maryland Assessment of Social Competence
100 to develop a 3-point scale that rates whether the subject displays efforts to extend the length of an interaction or facilitate the level of engagement. Use of the interlocutor's name, follow-up questions, eye contact, reactive affect, gestures, and intonation all indicate relatedness. The presence of several of these features yields a Relatedness score of 2 for the interaction, and their absence yields a score of 0. Interrater reliability was good for the Relatedness scale (kappa: 0.75; SE: 0.10), and stability across sessions was high (α: 0.84).
3. Behavioral Activity Level (ARS Code)
Time-budget approaches have demonstrated a clear relationship between activity level and clinical improvement in schizophrenia.
116,117 Engagement in social activities, employment, and activities of daily living (ADL) competence are key components of assessments of functioning.
118,119 In video examples, behavioral activity level was an obvious difference between subjects. The initiative, effort, goal-direction, and complexity of daily activities varied across subjects and across sessions. Behavioral activity level appeared salient to neurocognitive capacity and could be reliably tracked in the video.
We adapted an existing scale of activity level to rate behavioral activity in brief increments of video. The original scale
120,121 rates the activities in a diary the subject completes with an interviewer, scoring 4 time-periods in the day. It is scaled toward active, organized, and motivated behaviors, with subjects who participate in passive, solitary activities receiving lowest scores. Our adaptation, the 5-point Activity Rating Scale (ARS), rates the level of complexity and initiative required for subjects' behaviors seen in the video. We added activities commonly seen on video (e.g., phone calls) to the scale. The five points of the ARS scale are anchored by both specific behaviors (e.g., eating a meal, using a microwave) and behavioral descriptors (e.g., doing simple tasks one after the other, an isolated activity requiring some effort). The scale remains weighted toward complex and effortful behaviors and toward multitasking (ARS=4) and sequencing of behaviors (ARS=3). Time in which a subject cooks “from scratch” while fielding a phone call receives a 4; time in which a subject watches TV receives a 0. We give an ARS rating for each 15 minutes of video; 15-minute increments showed better range and interrater reliability than 10- or 20-minute increments.
The ARS scale captures the quantity of activity (e.g., how busy subjects were) and its organizational quality (e.g., whether tasks were sequenced together toward an explicit goal). For example, two subjects spent most of their first observed sessions in their neighborhoods. The first subject walked from class to the library to return a video and pick up another. Then he went across the street to pick up food for dinner. Then he walked to the bus stop and caught a bus home. In 45 minutes, he pursued tasks requiring some effort and initiative, and he also sequenced these tasks together to pursue them efficiently. He received ARS scores between 2 and 3 for the time, with a total score of 8. The second subject also spent 45 minutes walking in his neighborhood. He stopped at stores, often to say hello to a clerk, but he did not buy anything. He greeted those he recognized on the street. He decided which direction to walk in the moment, without a planned route or a final destination. Once, he yelled across the street to someone he knew. Although his walk through the neighborhood was as busy as the first subject’s, and whereas, superficially, he was engaged in the same tasks (e.g., going to the store), he lacked initiative and organization. He passively received an activity as it came into view (e.g., wandering into a store). He moved from one activity to the next without direction. As a result, this subject received ARS scores between 1 and 2 for the 45 minutes, with a total score of 4 (the score of 2 was the result of seeking to engage the company of the acquaintance across the street).
In fact, each 15 minutes is assigned 2 ARS scores, a highest and a predominant score, because some complex activities were extremely brief. If a subject is watching TV, but then makes a phone call and leaves a message, which takes 30 seconds, the subject receives a highest score of 2 and a predominant score (defined as the complexity of activity pursued for at least 7.5 minutes) of 0 for the 15 minutes. Use of the highest score increases the range of the scale used by coders and gives credit to subjects for short but complex behaviors. Yet, because it credits very brief behaviors within a background of inactivity, the predominant score is also used to reflect the overall level of activity. Both ARS highest (kappa: 0.77; SE: 0.09) and ARS predominant (kappa: 0.93; SE: 0.05) showed excellent interrater reliability and adequate alpha reliability across sessions (0.79 and 0.70 respectively).
4. Naturalistic Goal Pursuit (GOL Code)
Finally, we developed the goal-pursuit scale (GOL) to describe time use over longer blocks of time than the ARS scale and to indicate whether subjects actively pursued certain experiences. The GOL scale provides a global assessment of subjects’ time use. Although the skills tracked with the GOL scale reflect categories of functioning, cognitive skills—such as executive functioning,
122 metacognition,
123 social cognition,
124 and verbal memory
125—could play substantial roles in the pursuit of these activities. The GOL scale is modeled on time-budget approaches.
126–128 We categorize the activity seen on video into one of six domains denoting the type of activity (socializing, leisure, spirituality, activities of daily living, health, and vocation/education). The domains were adapted from the Activity Pattern Indicators scale
129,130 and from a taxonomy developed by Bogardus and colleagues.
131 In addition to these six goal domains, subjects occasionally pursued no goal, such as when napping or conversing only with the ethnographer. A seventh category—no goal pursuit—was created for those time-blocks. Multiple raters showed good reliability (84.6% agreement; SE 0.06) in assigning goal domains.
In addition to the goal domain, blocks of time are rated as
pursued or
received, denoting whether subjects passively take part or actively seek out the activity. In this sense, like the ARS scale, the GOL scale credits motivated activities. For example, time during which a subject travels from home to attend church receives a GOL code of “spirituality; pursued;” time during which a subject sits on the porch at a group-home talking with peers receives a GOL code of “socializing; received.” This emphasis is consistent with self-determination theory, which states that facilitating internal motivation fosters competence and potential;
132 it is also consistent with mental health and medical interventions
133 that aim to foster autonomy and competence. For these reasons, we use the GOL code to distinguish between activities that are a response to stimuli in the immediate environment (received) and those that reflect an intentional effort to expand beyond those stimuli (pursued). Interrater reliability was excellent in distinguishing pursued versus received goals (92.3% agreement; SE: 0.04), and session-to-session stability was adequate (α: 0.59).