Text Message Reminders of Appointments: A Pilot Intervention at Four Community Mental Health Clinics in London
Abstract
Objective:
Forgetting is commonly stated as a reason for missing mental health appointments. The authors examined the effect of short message service (SMS), or text message, reminders on the attendance of appointments at four community mental health clinics in London.
Methods:
Attendance of outpatient appointments roughly between March and June of 2008 (N=648), 2009 (N=1,081), and 2010 (N=1,088) was examined. Reminder messages were sent seven and five days before an appointment in 2009 and seven and three days before an appointment in 2010; patients in the 2008 sample received no reminder messages. Appointment attendance during the sample periods was compared by using multiple logistic regression analysis and adjusting for sociodemographic and clinical confounders.
Results:
Missed appointments accounted for 36% of appointments in 2008, 26% of appointments in 2009, and 27% of appointments in 2010. The relative risk reduction in failed attendance was 28% between the 2008 and 2009 samples and 25% between the 2008 and 2010 samples. Attendance rates were significantly higher for the 2009 and 2010 samples than for the 2008 sample (p<.001) but did not differ between the two intervention periods.
Conclusions:
SMS-based technology can offer a time-, labor-, and cost-efficient strategy for encouraging engagement with psychiatric outpatient services. In England alone, a reduction of 25% to 28% in missed outpatient clinic appointments would translate to national cost savings of more than £150 million, or $245 million, per year, and likely have clinical benefits as well. (Psychiatric Services 63:161–168, 2012; doi: 10.1176/appi.ps.201100211)
Missed outpatient clinic appointments cost the National Health Service (NHS) in England more than £600 million ($980 million) a year for a population one-fifth the size of the United States (1–4). Mental health services are among the worst affected, given that 20% to 34% of scheduled appointments result in “did-not-attend” (DNA) outcomes (5,6). Missed psychiatric appointments delay access to vital care and increase the risk of continuing nonattendance, disengagement from services, rehospitalization, and medication nonadherence (6–10).
Because forgetting is a commonly stated reason for missing psychiatric appointments (6,9,11), attempts to encourage attendance have frequently relied on reminders. Simple postal or telephone-call prompts, usually delivered one to 14 days before the psychiatric appointment, have been reported to reduce rates of DNA by 26% to 76% (5,12–15), particularly when they are received close to the time of the appointment (16).
An alternative approach has relied on short message service (SMS) technology, or text messaging. SMS messages are more cost-effective, are swifter to deliver, and are likely to receive a faster response than letters, and they are less intrusive than phone calls. Substantial numbers of messages may be dispatched simultaneously, reducing labor expenditure. Text message interventions have been successfully used in diverse medical settings to improve clinic attendance, health care-related communication, medication adherence, and clinical outcomes for an extensive range of medical conditions, including diabetes, tuberculosis, and HIV and other sexually transmitted infections (2,17–31).
No systematic investigation has explored the utility and suitability of SMS appointment reminders for mental health patients, with the exception of a small pioneering study by Pijnenborg and colleagues (32). Therefore, robust evaluations are needed before undertaking a wider rollout of SMS technology in mental health settings (1).
We report a pilot study of 2,817 outpatient appointments in four community mental health clinics located in an inner borough of London. The study is the largest investigation of psychiatric appointment reminders reported to date and the first large systematic exploration of SMS-based applications in mental health services. We compared the attendance outcomes of appointments sampled from the same three-and-a-half-month period during 2008 (N=648), 2009 (N=1,081), and 2010 (N=1,088). In 2008, patients received no reminders; in 2009, they received SMS reminders seven and five days before appointments; and in 2010, they received SMS reminders seven and three days before appointments. We hypothesized that rates of nonattendance would be significantly reduced during the 2009 and 2010 interventions compared with rates for 2008 and that the greatest reduction in missed appointments would occur in 2010.
Methods
Analytic sample
The analytic sample comprised 2,817 appointments for 1,256 outpatients in four clinics run by the Assessment and Shared Care (ASC) Community Mental Health Team of the Oxleas NHS Foundation Trust between March 17 and July 2 of 2008, March 16 and July 2 of 2009, and March 16 and July 2 of 2010. All appointments in the 2009 and 2010 samples were prompted by SMS reminders to outpatients' mobile phones seven and five days and seven and three days, respectively, before each appointment. No reminders took place during the 2008 study period, which was used for comparison. (No reminders were issued between May 11, 2009, and May 28, 2009, because of a technical fault, and data for 2008, 2009, and 2010 during that period were omitted from the analyses.)
The ASC team is based at the Ferryview Health Center in Woolwich and provides outpatient mental health care for residents of Greenwich, an inner-London borough with a population of about 228,100. The four clinics include a postdischarge clinic for patients recently discharged from inpatient care, an assessment clinic for patients referred by a general practitioner or other health care professional, and two outpatient (morning and afternoon) clinics for patients who have an established relationship with the service and attend standard appointments within a treatment course. The four clinics are run by a variety of psychiatrists, from consultants to senior house officers.
Appointments were included in the sample if the electronic clinical records system of the Oxleas Trust, a Web-based application managed by British Telecom and called “RiO,” contained a mobile phone number valid in the United Kingdom for the patient; if the appointment outcome was recorded as “attended” or DNA by administrative and reception staff under the direction of the patient's health care provider; if the patient had an ICD-10 (33) diagnosis of a mood or affective disorder (F301, F309–F323, F328–F334, and F339–F341), of a neurotic, stress-related, or somatoform disorder (F400–F402, F409–F412, F418–F422, F429–F432, F444–F445, F448–F450, F452–F454, and F480), of schizophrenia or a schizotypal or delusional disorder (F200, F202–F206, F209, F21x, F220, F229–F233, F238–F239, F24x, F250–F252, F259, and F29x), of a disorder of adult personality or behavior (F600–F609, F61x, F620, F630, F633, F641, F648, and F688), or of a mental or behavioral disorder due to psychoactive substance use (F100–F103, F105, F107, F109, F112, F119–F122, F125, F129–F130, F132, F139–F142, F144, F149–F151, F167, F171, F190–F192, F195, F197, and F199); and if the patient consented to receive SMS reminders.
SMS intervention
The study began in March 2009 as a nonrandomized pilot intervention by the ASC team at Ferryview to encourage appointment attendance among outpatients. The intervention gave rise to encouraging results and was repeated with a modified protocol from February 2010 onward.
For the 2009 pilot, all the electronic clinical records in RiO were downloaded daily to a structured query language database. Scripts were run to extract the participants' mobile phone numbers and time and date of appointments and to “cleanse” the data by removing spaces, dashes, letters, and numbers with fewer or more than 11 digits or those not starting with 07—all valid mobile phone numbers in the United Kingdom have 11 digits and start with 07. Next, e-mails with the “clean” details were generated to the internal e-mail server of the Trust and forwarded to a different server attached to an SMS gateway. The SMS gateway converted the e-mails into mobile-phone SMS messages, which were sent to the recipients via a global-system-for-mobile-communications modem.
The reminders were received at 6 p.m., seven and five days before each scheduled appointment, and contained the following standardized script: “You have an appointment on TESTDAY at TESTTIME at Ferryview. Please call TESTNUMBER if you cannot come.” In line with NHS Trust confidentiality procedures, the patient and the Trust were not identified in the SMS. The cost of the reminders was the same as for standard SMS messages.
During the 2010 pilot an identical protocol was used except that the internal e-mail server of the NHS Trust generated encrypted emails to an NHS.net account set up for the intervention that used scripts that were configured with extra software to comply with NHS.net security requirements. (NHS.net mail is a national secure e-mail service endorsed by the British Medical Association and the Royal College of Nursing.) The NHS.net service forwarded the e-mails to its own SMS gateway, which generated the SMS reminders. The reminders were delivered seven and three days before each appointment at no cost to the Oxleas Trust.
Quality data checks and patient consent
The Trust's clerical and administrative staff updated the outpatients' details in RiO regularly. To ensure that as many clients as possible would receive the SMS intervention, we (DH, HS, or HS) contacted outpatients individually to validate, add, or update their mobile phone number entries in RiO. If no valid mobile number was available, a home, work, or other alternative phone number that had been entered in Rio or obtained from the client's general practitioner was used to contact the patient in order to request a mobile phone number. Alternatively, reception staff were asked to record the patient's mobile phone number at the next outpatient appointment. All ICD-10 diagnostic codes in RiO were confirmed by the patients' psychiatrists before the analysis.
Consultation with three research ethics committees—Bexley and Greenwich National Research Ethics Committee; Oxleas Research and Development Office, Joint South London and Maudsley; and Institute of Psychiatry Research Ethics Committee—confirmed that the SMS intervention met criteria for service development and improvement and did not require ethical approval. However, in line with the Trust's policies on confidentiality and information sharing, the Oxleas Informational Governance Manager advised that patients should give consent to participate in the intervention. For the 2009 pilot, consent was obtained when patients were contacted to validate, add, or update their mobile number entries and was recorded in RiO. For the 2010 pilot, all patients were sent an SMS message before the launch of the intervention that informed them about the service and instructed them to contact ASC if they did not wish to receive SMS reminders.
Statistical analysis
The statistical analysis was conducted using SPSS, version 15 for Windows (34), and Stata, version 10.0 for Windows (35). To account for seasonal effects (36), our main statistical analysis was restricted to appointments scheduled for the same three-and-a-half-month period in 2008, 2009, and 2010 (N=2,817).
The subjects of the measurements were appointments rather than individual patients; therefore, the database contained as many copies of each patient's characteristics as the total number of appointments scheduled for that patient. To account for the nonindependence of measurements related to different appointments for the same patient—in other words, each patient was a cluster—our main statistical analysis was adjusted for clusters throughout. Cluster-adjusted regression analysis provides robust variance estimates accounting for within-cluster correlations and safeguards against potential violations of the standard regression assumptions (37).
Differences across the three study periods in service and patient characteristics, such as missed appointments, clinic, patient gender, age, ethnicity, occupation, marital status, diagnosis, number of appointments or of prior inpatient admissions, and length of last inpatient admission were examined by using univariable linear, logistic, or multinomial regression. To examine the effect of the SMS intervention on attendance outcome, we identified patient and service characteristics other than the SMS reminders that might influence attendance outcome to control for potential confounders in the main analysis. In this preliminary step, we carried out a series of exploratory logistic regression analyses, each examining the effect of a separate study variable on attendance outcome for all sampled appointments during 2008–2010, adjusting for the three levels of SMS intervention (none, seven- and five-day reminders, and seven- and three-day reminders. Variables that emerged as significant predictors of attendance in these exploratory analyses were entered as covariates in the final model—a multiple logistic regression analysis examining the effect of the SMS intervention on attendance outcome and controlling for potential confounders.
Using segmented regression analysis adjusting for autocorrelation and seasonality, we also conducted interrupted-time-series analysis on all appointments scheduled for the continuous period March 2008–July 2010 regardless of diagnosis, mobile phone ownership, and appointment outcome (N=24,709) to examine whether changes in intervention status (off-on-off, etc.) were associated with the rate of appointments missed or attended. This approach aimed to address potential problems associated with unknown changes in clinic practices, such as how administrative staff classified appointment outcome, and to identify trends in missed or attended appointments over time. The proportion of appointments missed or attended every 15 days was used as the outcome.
Results
Of all ASC appointments (N=24,709) scheduled between March 2008 and July 2010, 70% (N=17,391) were classified as attended or DNA; 14% (N=3,526) were cancelled by the clinic or clinician, 12% (N=2,900) were cancelled by the patient, and 4% (N=892) were entered in error. Seventy percent of appointments (N=17,383) were for patients who had a mobile phone number valid in the United Kingdom, and 74% (N=18,329) were for patients who had been diagnosed as having an ICD-10 diagnosis that met criteria for inclusion in the sample. Only 19 (.1%) patients opted not to receive SMS. Of the appointments included in the main statistical analysis (N=2,817), 882 were for patients with a mood or affective disorder; 666 were for patients with a neurotic, stress-related, or somatoform disorder; 458 were for patients with schizophrenia or a schizotypal or delusional disorder; 452 were for patients with a disorder of adult personality or behavior; and 359 were for patients with mental or behavioral disorders due to psychoactive substance use.
Table 1 presents the number of appointments scheduled or missed during the three study periods by characteristic of the patients or by ASC clinic. Table 2 lists patient characteristics during the three study periods.
The percentage of appointments that resulted in DNA outcomes was lower during the first (26%) and second (27%) SMS interventions than during the baseline period (36%) (Table 1). The absolute- and relative-risk reductions between baseline and the first intervention and between baseline and the second intervention were 10% and 28%, and 9% and 25%, respectively. Before covarying for potential confounders was undertaken, the three study periods differed significantly in attendance outcome (p<.001), patient diagnostic characteristics (p=.005), and clinic holding the appointments (p<.001) (Table 1); they also differed in patient age (p<.001) (Table 2).
With the exception of gender and length of last inpatient admission, all study variables emerged as significant predictors of appointment attendance. They included the clinic where the appointment was made (χ2=15.69, df=3, p=.001), patient diagnosis (χ2=58.24, df=4, p<.001), number of appointments per patient (odds ratio [OR]=1.02, 95% confidence interval [CI]=1.00–1.03, p=.03), number of prior inpatient admissions (OR=.88, CI=.82–.94, p<.001), age (OR=1.03, CI=1.02–1.04, p<.001), race-ethnicity (χ2=8.20, df=3, p=.04), occupation (χ2=11.75, df=4, p=.02), and marital status (χ2=12.05, df=3, p=.007). The results of the final multiple regression model are presented in Table 3. After adjusting for the aforementioned confounders, we found that the effect of the SMS intervention on appointment attendance was highly significant (p<.001). Attendance rates during both intervention periods were significantly higher than during the comparison period but did not differ from each other (OR=1.14, CI=.92–1.41, p=.22). Our primary analysis included only appointments that resulted in attended or DNA outcomes, raising the possibility that reminders simply led to more cancellations. Table 4 presents the numbers and percentage of appointments that resulted in each of four different outcomes (attended, missed, cancelled by patient, or cancelled by clinic or clinician) in the three study periods. Although similar percentages of appointments were cancelled by clients across 2008–2010, a relatively high percentage of cancellations by clinic or clinician was observed in 2010.
The results of the interrupted four-season autoregressive model (SARIMA [1, 0, 0, 4] as best fit), presented in Table 5, support those of the main logistic regression conducted on the restricted sample data. There was a strong decline, by 9% monthly or by 5% every 15 days, in the proportion of missed appointments after the interventions (p=.03). Similarly, there was a strong monthly improvement of 11% in the proportion of appointments attended after the interventions (p=.01).
Discussion
A simple cost-effective delivery of SMS reminders seven and five days or seven and three days before scheduled appointments resulted in a 25% to 28% relative risk reduction in failed attendance at four community mental health clinics in an inner borough of London. The PROMPTS randomized controlled study in Leeds, United Kingdom, reported a closely similar relative risk reduction, 26%, in response to a short orientation letter received by patients three days before a first psychiatric appointment (14). Although the relative risk reductions reported by the two studies fell within the lower end of the ranges reported in the literature (26% to 76%) (5,12–15,38), both studies were significantly larger and more rigorous than other investigations.
Contrary to our hypothesis, reminders delivered in 2010 seven and three days before appointments did not result in a greater reduction in missed appointments than reminders delivered in 2009 seven and five days before the appointments. However, given that the two delivery frames differed by only two days, it would be interesting to examine whether a greater contrast in the timing of reminders would show significant differences in reduction of DNA outcomes (16). Although the percentage of missed appointments was consistently low in both intervention periods compared with the percentage in 2008, an increase in attendance was more clearly observed in 2009 than in 2010, when there was a pronounced increase in cancellations by the clinic or clinician.
The annual direct cost of missed outpatient appointments at hospital clinics in England is estimated to exceed £600 million, or $980 million (2,4). Between 2007 and 2008, 6.5 million appointments were missed, and hospitals lost nearly £100, or $163, per patient in revenue (4). If achieved across the United Kingdom, a relative risk reduction in failed attendance equivalent to the reduction reported in this study would result in national cost savings of well over £150 million, or $245 million, per year (2) and, most likely, in substantial clinical benefits (6,7,9,10).
Our SMS intervention required minimal investment in information technology infrastructure. The procedure for sending automated SMS was easy to link with existing electronic patient records in the hospital administrative database. In addition, because of the automation of the system, no added labor or staff training was necessary. Notably, using the NHS.net service in the 2010 pilot eliminated any costs associated with sending reminders. It would appear, therefore, that SMS technology can offer a time-, labor-, and cost-efficient strategy for encouraging attendance of psychiatric outpatient services.
This study used an innovative means of addressing a widespread and important problem and, therefore, is of direct clinical importance. It is the largest investigation to date of appointment prompts for psychiatric outpatients. With the exception of a small pioneering study by Pijnenborg and colleagues (32), it is also the first to use SMS-based technology with psychiatric outpatients. In addition, the inclusion criteria in the three study periods eliminated seasonal effects and the effect of confounders associated with mobile phone ownership. The thorough and systematic statistical approach identified several additional confounders and then simultaneously controlled for them in the main regression model. Finally, our alternative statistical analysis increased the generalizability of findings by including all psychiatric outpatient appointments in a large NHS Foundation Trust in England over a period of 2.3 years.
Our study also had limitations. The lack of randomization is an obvious limitation of our design, even though the robust sample size and use of stringent sampling and statistical controls for seasonal, diagnostic, clinical, service, and demographic confounders reduced the risk of unknown systematic differences across the compared study groups. A randomized, controlled trial would have provided a more rigorous approach to attaining our study goals. In the future, therefore, such a trial is warranted to replicate and extend our findings.
Another limitation of the study is that the two intervention periods could be compared with only one other period, in 2008, because RiO data are not available before October 1, 2007. However, our alternative analysis of the unrestricted data that corrected for time trends supported the results of the main analysis. Finally, it should be emphasized that the pilot nature of this work calls for additional studies to better capture the ways that SMS may influence attendance.
Conclusions
SMS reminders delivered seven and five days and seven and three days before scheduled psychiatric appointments effectively reduced rates of nonattendance in four community mental health clinics in an inner borough of London. The rate of reduction (25% to 28%), combined with the low cost of the SMS intervention and the high cost of missed outpatient appointments, suggests that a large-scale rollout of SMS-based systems in health contexts is likely to result in clinical benefits and substantial cost savings. In England alone, a 25% to 28% reduction in missed outpatient appointments would translate to national cost savings of more than £150 million, or $245 million, per year.
The potential utility of SMS-based approaches for encouraging appointment attendance among psychiatric outpatients relies on the assumption that mobile phone use is widespread in this population, and that SMS reminders are well received. Both assumptions were supported by this study and earlier studies (39,40).
Acknowledgments and disclosures
The authors thank the board of directors of the Oxleas NHS Foundation Trust; Tristan David, B. Eng., and Rob Richardson of Sector IT Solutions for technical assistance; and the staff and patients of the ASC team at the Ferryview Health Centre in Woolwich, England, for their support, participation, and feedback.
The authors report no competing interests.
References
1.
Car J, Ng C, Atun R, et al.: SMS text message health care appointment reminders in England. Journal of Ambulatory Care Management 31:216–219, 2008
2.
Atun RA, Sittampalam SR, Mohan A: Uses and Benefits of SMS in Health Care Delivery. London, Imperial College London, Tanaka Business School, Centre for Health Management, 2005
3.
Hennessy PM, Kite M: Missed appointments cost NHS £575m a year. Sunday Telegraph London, Aug 21, 2005
4.
“No shows” cost the NHS millions. BBC News, Aug 12, 2009. Available at www.news.bbc.co.uk/1/hi/health/8195255.stm
5.
Magnes RM: Outpatient appointments: a necessary evil? A literature review and survey of patient attendance records. Psychiatric Bulletin 32:458–460, 2008
6.
Mitchell AJ, Selmes T: Why don't patients attend their appointments? Maintaining engagement with psychiatric services. Advances in Psychiatric Treatment 13:423–434, 2007
7.
Appleby L, Shaw J, Amos T, et al.: Suicide within 12 months of contact with mental health services: national clinical survey. British Medical Journal 318:1235–1239, 1999
8.
Chen A: Noncompliance in community psychiatry: a review of clinical interventions. Hospital and Community Psychiatry 42:282–287, 1991
9.
Killaspy H, Banerjee S, King M, et al.: Prospective controlled study of psychiatric outpatient nonattendance: characteristics and outcome. British Journal of Psychiatry 176:160–165, 2000
10.
Nelson EA, Maruish ME, Axler JL: Effects of discharge planning and compliance with outpatient appointments on readmission rates. Psychiatric Services 51:885–889, 2000
11.
Sparr LF, Moffitt MC, Ward MF: Missed psychiatric appointments: who returns and who stays away. American Journal of Psychiatry 150:801–805, 1993
12.
Dockery F, Rajkumar C, Chapman C, et al.: The effect of reminder calls in reducing nonattendance rates at care of the elderly clinics. Postgraduate Medical Journal 77:37–39, 2001
13.
Jayaram M, Rattehalli RD, Kader I: Prompt letters to reduce nonattendance: applying evidence based practice. BMC Psychiatry 8:90, 2008
14.
Kitcheman J, Adams CE, Pervaiz A, et al.: Does an encouraging letter encourage attendance at psychiatric outpatient clinics? The Leeds PROMPTS randomized study. Psychological Medicine 38:717–723, 2008
15.
Rusius CW: Improving outpatient attendance using postal appointment reminders. Psychiatric Bulletin 19:291–292, 1995
16.
Reda S, Rowett M, Makhoul S: Prompts to encourage appointment attendance for people with serious mental illness. Cochrane Library, Apr 14, 2010. Available at onlinelibrary.wiley.com/doi/10.1002/14651858.CD002085/full
17.
Geraghty M, Glynn F, Amin M, et al.: Patient mobile telephone “text” reminder: a novel way to reduce nonattendance at the ENT outpatient clinic. Journal of Laryngology and Otology 122:296–298, 2008
18.
Koshy E, Car J, Majeed A: Effectiveness of mobile-phone short message service (SMS) reminders for ophthalmology outpatient appointments: observational study. BMC Ophthalmology 8:9, 2008
19.
Krishna S, Boren SA: Diabetes self-management care via cell phone: a systematic review. Journal of Diabetes Science and Technology 2:509–517, 2008
20.
Kelly JD, Giordano TP: Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 25:1137, 2011
21.
Liang X, Wang Q, Yang X, et al.: Effect of mobile phone intervention for diabetes on glycaemic control: a meta-analysis. Diabetic Medicine 28:455–463, 2011
22.
Pop-Eleches C, Thirumurthy H, Habyarimana JP, et al.: Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 25:825–834, 2011
23.
Kristjansdottir OB, Fors EA, Eide E, et al.: Written online situational feedback via mobile phone to support self-management of chronic widespread pain: a usability study of a Web-based intervention. BMC Musculoskeletal Disorders 12:51, 2011
24.
Cooper G, Walker J, Harris D, et al.: Use of text messaging to audit early clinical outcome following vasectomy in primary care. British Journal of General Practice 61:280–282, 2011
25.
Menon-Johansson AS, Cohen CE, Jones R, et al.: Interventions to increase access to STI services: a study of England's “high-impact changes” across three central London clinics. Sexually Transmitted Infections 86:540–544, 2010
26.
Choi J, Bakken S: Web-based education for low-literate parents in a neonatal intensive care unit: development of a website and heuristic evaluation and usability testing. International Journal of Medical Informatics 79:565–575, 2010
27.
da Costa TM, Salomao PL, Martha AS, et al.: The impact of short message service text messages sent as appointment reminders to patients' cell phones at outpatient clinics in Sao Paulo, Brazil. International Journal of Medical Informatics 79:65–70, 2010
28.
Neville RG, Reed C, Boswell B, et al.: Early experience of the use of short message service (SMS) technology in routine clinical care. Informatics in Primary Care 16:203–211, 2008
29.
Fairhurst K, Sheikh A: Texting appointment reminders to repeated nonattenders in primary care: randomised controlled study. Quality and Safety in Health Care 17:373–376, 2008
30.
Waller A, Franklin V, Pagliari C, et al.: Participatory design of a text message scheduling system to support young people with diabetes. Health Informatics Journal 12:304–318, 2006
31.
Lewisham cuts missed appointments with SMS. E-Health Insider, Sept 10, 2007. Available at www.ehi.co.uk/news/primary-care/3014
32.
Pijnenborg GH, Withaar FK, Brouwer WH, et al.: The efficacy of SMS text messages to compensate for the effects of cognitive impairments in schizophrenia. British Journal of Clinical Psychology 49:259–274, 2010
33.
ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research. Geneva, World Health Organization, 1993
34.
Apache Software Foundation: SPSS, v 15 for Windows. Chicago, SPSS, 2008
35.
STATA v 10.0 for Windows. College Station, Tex, Stata, 2007
36.
Mitchell AJ, Selmes T: A comparative survey of missed initial and follow-up appointments to psychiatric specialties in the United Kingdom. Psychiatric Services 58:868–871, 2007
37.
Williams RL: A note on robust variance estimation for cluster-correlated data. Biometrics 56:645–646, 2000
38.
Rajasuriya M, De Silva V, Hanwella R: Effectiveness of reminders in reducing nonattendance among outpatients. Psychiatrist 34:515–518, 2010
39.
Milne RG, Horne M, Torsney B: SMS Reminders in the UK National Health Service: an evaluation of its impact on “no-shows” at hospital outpatient clinics. Health Care Management Review 31:130–136, 2006
40.
Sanghara H, Kravariti E, Jakobsen H, et al.: Using short message services in mental health services: assessing feasibility. Mental Health Review Journal 15:28–33, 2010
Figures and Tables
Information & Authors
Information
Published In
Cover: Cups 4 Picasso, by Jasper Johns, 1972. Lithograph composition.
Gift of Celeste Bartos, the Museum of Modern Art, New York. © Jasper Johns
and ULAE/Licensed by VAGA, New York. Digital image © The Museum of Modern
Art/Licensed by SCALA/Art Resource, New York.
History
Published online: 1 February 2012
Published in print: February 2012
Authors
Metrics & Citations
Metrics
Citations
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
View Options
View options
PDF/EPUB
View PDF/EPUBLogin options
Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.
Personal login Institutional Login Open Athens loginNot a subscriber?
PsychiatryOnline subscription options offer access to the DSM-5-TR® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.
Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).