Ahmed Elkashef1, Layla Alhyas1, Hanan Al Hashmi1, Dina Mohammed1, Alison Gonzalez1, Rinso Paul1, Shamil Wanigaratne1, Tarek Gawad1, Richard Rawson2, Ali Al Marzooqi1 and Hamad Al Ghaferi1
مقاييس أداء البرامج بالمركز الوطني لإعادة التأهيل بالإمارات العربية المتحدة، 2013
أحمد الكاشف، ليلى الهياس، حنان الهاشمي، دينا محمد، أليسون جونزاليس، رينزو بول، شاميل وانيجراتنه، طارق جواد، ريتشارد روسون، علي المرزوقي، حمد الغافري
الخلاصة: تهدف هذه الدراسة إلى قياس أداء البرامج بالمركز الوطني لإعادة التأهيل والخدمات التي يقدمها المركز. وأُعِدت أداة مكونة من ست نقاط استناداً إلى المعايير الدولية وتم توفير مقاييس الأداء المناسبة لها. وطُبِّقت عناصر الأداة على ملفات البيانات الإلكترونية لعام 2013. وأشارت النتائج إلى وجود بعض القضايا التشغيلية والقضايا التي تتعلق بالجودة ونظام البيانات. وبينما تيسَّر الوصول إلى المعلومات الخاصة بالطاقة الاستيعابية ووقت الانتظار ومشاركة الأسر، كان من الصعب تصنيف معلومات أخرى والوصول إليها، في حين لم تتوفر بعض المعلومات الأخرى في النظام الإلكتروني وتعيَّن استرجاعها من مصادر أخرى، مثل المعلومات حول فعالية البرامج. وارتفع معدل عدم الحضور في المواعيد المحددة (%46)، ومعدل الدخول إلى المركز مرة ثانية (%52) ولم تحضر معظم الأسر الجلسات الأسرية (%72) وكانت الدراسة مفيدة للغاية إذ حددت الثغرات في العمليات والسجلات. ووزِّعت النتائج على الفِرق المختلفة للاستفادة منها في تحسين جودة البيانات والخدمات، وسوف تُستَخدم الأداة في عمليات التقييم السنوي.
ABSTRACT The aim of this study was to measure the performance of the National Rehabilitation Center (NRC) programme and the services it provides. A 6-point tool was developed based on international standards with appropriate performance measures. The elements of the tool were applied to the electronic data files for 2013. The results showed that there were some operational, quality and data system issues. Some items were easily accessible, e.g. capacity, waiting time, family involvement. Others were difficult to sort and find and some were not available in the electronic system and had to be retrieved from other sources, e.g. programme effectiveness. There was a high no-show rate for appointments (46%) and readmission rate (52%) and most families did not attend family sessions (72%). This was a valuable exercise which identified gaps in operations and records. The findings were shared with the different teams to help improve the quality of data and services and the tool will be used for annual performance evaluations.
Mesure de la performance du programme du Centre national de réadaptation aux Émirats arabes unis, 2013
RÉSUMÉ : La présente étude avait pour objectif de mesurer la performance du programme du Centre national de réadaptation et les services qu'il fournit. Un outil en six points comportant des mesures de la performance a été élaboré sur la base des normes internationales. Les éléments de l'outil ont été appliqués aux fichiers de données électroniques pour l’année 2013. Les résultats ont montré certains problèmes aux niveaux des opérations, de la qualité et du système de données. Certaines informations étaient facilement accessibles (capacité, temps d’attente, implication familiale), contrairement à d’autres informations difficiles à obtenir (efficacité du programme) ou qui n’étaient pas disponibles dans le système de données électroniques et devaient être obtenues via d’autres sources. Il y avait un taux élevé de non-présentation aux rendez-vous (46 %) et de réadmission (52 %) ; la plupart des familles n'assistaient pas aux séances qui leur étaient dédiées (72 %). L’exercice s’est révélé très instructif et a permis d’identifier des lacunes dans les opérations et les dossiers. Les conclusions ont été portées à la connaissance des différentes équipes de façon à améliorer la qualité des données et des services et l'outil sera utilisé pour les évaluations annuelles de la performance.
1The National Rehabilitation Center, Abu Dhabi, United Arab Emirates (Correspondence to: Ahmed Elkashef:
Received: 15/06/16; accepted: 01/11/16
Introduction
The National Rehabilitation Center (NRC) is the main addiction treatment facility in the United Arab Emirates. It provides data on the programme performance on an annual basis to its monitoring board. In order to reach consensus on what tools to use for the evaluation of our programme, we reviewed the available data from the Guiding Principles and Elements of Recovery-oriented Systems of Care (1), the Treatment Outcomes Profile data of the National Drug Treatment Monitoring System in the United Kingdom (2), and the Drug Evaluation Network System (DENS) (3). As highlighted by Garnik and colleagues, much research is still needed to develop new performance measures that are “important, scientific and feasible” (4). Our paper focuses on the development of 3 types of measures—structural, process and outcome measures—as well as composite measures that include multiple domains.
Methods
Several meetings were held with the research team, the clinical team and the data group at the NRC to discuss and review the available data and to come to consensus on items for programme performance measures that are meaningful, clearly defined and readily accessible in the electronic data system. We developed a 6-point tool—the NRC Program Performance Scale—which incorporated some of the items and guidelines from the reviewed data in order to evaluate the treatment services and to monitor progress and treatment effectiveness. The tool includes both structural and process measures, which are outlined below.
1. Access to treatment: This is a measure of the capacity and overload of the treatment facility, assessed by:
Number of contacts made by patients requesting treatment per year
Number of no-shows (patients who booked an appointment but did not show up for it)
Average waiting time for admitting patients to treatment.
2. Use of evidence-based practices: This measures the different types of treatment programme delivered and attempts to compare their effectiveness with a focus on:
Number of patients referred to each programme and the number who completed each programme
Use of addiction medications when indicated.
3. Family engagement: A very important component of a successful outcome is engaging families in the patient treatment. Family therapy has been shown to increase retention in treatment, reduce drug use (5), improve relationships and reduce domestic violence (6). Family therapy among patients released from prison was also found to reduce parole violations and relapse (7). This measure includes:
Number of family sessions recommended by the treatment team and the number of sessions attended
Relation between the number of family sessions attended and retention in treatment.
4. Treatment completion: This measure includes:
Number of patients completing the different phases of treatment
Number of patients transferred between different phases of treatment
Drop-out rates in each phase of treatment and reasons for drop-out
Number of readmissions.
5. Participation in continuing care: This measures the connectivity of different organizations that provide services to patients for their long-term treatment, e.g. vocational programmes, social welfare, housing, education and health. In essence, this a measure of how the different organizations operate as a recovery-oriented system of care. The NRC is primarily a treatment organization, so we tried to determine 2 measures:
Number of patients referred by the NRC to different organizations based on their long-term treatment needs
Number of patients receiving long-term treatment at the NRC, e.g. attending self-help groups and patients who come for their follow-up medication appointments at the outpatient clinic.
6. Satisfaction with treatment: This measure is derived from a survey that was answered by the patients at the end of treatment asking for their feedback on different components of the services provided, starting from initiating the call to discharge from treatment.
Data were collected from the electronic medical records of all NRC patients for the year 2013 using a data collection template created in Excel. Following the completion of data collection, the data were revised, cleaned and edited as necessary for statistical analysis. Descriptive statistics were used as appropriate.
Results
Patient number and characteristics
A total of 632 new patients were admitted for treatment at the NRC between January 2013 and December 2013. Of these, 460 (73%) were admitted voluntarily and 172 (27%) through the legal system. The age of the patients ranged from 14 to 67 years, with a mean age of 29 (SD 9.3) years. The majority of the patients (98%) were males, 56% resided in Abu Dhabi, 61% were single, 70% had middle or secondary school education and 68% used more than one substance.
Programme outcomes
Access to treatment
A total of 850 people requested voluntary appointments in 2013 for either new assessment [559 (66%)] or re-assessment if they had been admitted before to the NRC, 291 (34%). All 850 subjects were booked for appointments, however only 460 showed up.
The average waiting time from contacting the centre for an appointment and having the appointment was 6.9 days: 493 (58%) patients waited between 1 and 10 days, 173 (20%) had appointments scheduled on the same day and 184 (22%) waited 10 days or more for an appointment.
Use of evidence-based practices
Some programmes were listed as such in the electronic system, e.g. matrix programme (a group therapy programme based on cognitive behavioural therapy and relapse prevention), while others were not, e.g. contingency management. The data listed in the system and the number of sessions attended by patients are presented in Figure 1. Meditation, reflection, 12-step group, interpersonal therapy and matrix were the top 5 programmes/activities attended by the patients. Data on patient enrolment, completion and drop-out from each programme were not available in the system so we could not compare the effectiveness of each as planned.
Data on medications prescribed per patient were not available in the system; however the number of prescriptions for each medication for 2013 was obtained from the pharmacy (Figure 2). Lofexidine (0.2 mg tablets) was the most prescribed medication in 2013, mainly for opiate withdrawal, followed by bupronorphine (8 mg SL film).
Family involvement
Most of the families of the 632 patients admitted, 456 (72%), did not attend any family sessions, 125 (20%) attended 1-3 sessions and 51 families (8%) attended more than 3 sessions. Family attendance varied somewhat by the patient’s age. We expected to see more family sessions among younger patients, however more families attended 1-3 sessions in the older patient age group > 50 years (35%) compared with 16% in the younger age group < 20 years.
Treatment completion
Most patients were admitted initially to either the detox unit (30%) or outpatient services (49%) as shown in Figure 3.
Detox programme: In 2013, 436 patients received detox treatment. Of these, 58 (13%) completed detox and were discharged home on their request, 169 (39%) completed detox and were transferred to inpatient services and 61 (14%) completed detox and were transferred to the outpatient services. About a third, 148 (34%), dropped out, left against medical advice or were discharged for disruptive behaviour.
Outpatient programme: A total of 552 patients received treatment from the outpatient programme in 2013. Of these, 194 (35%) dropped out, left against medical advice or had a disciplinary discharge, 162 (30%) transferred to inpatient services, 13 (2%) completed treatment and were discharged and 183 (33%) were still continuing in the programme by the end of the year.
To study the possible reasons for dropping out of these programmes, we examined the family visits among those who dropped out. We also looked at place of residence as a possible contributing factor because almost half of the NRC patients lived in other Emirates, not Abu Dhabi, which required long travelling times. For the detox programme, families did not attend any family sessions in 133 (90%) of the 148 patients who dropped out. For the outpatient programme, families did not attend any sessions in 177 (91%) of the 194 patients who dropped out and for 11 (6%) patients, their families attended 1 session. There was no association between living outside of Abu Dhabi city and drop-out for either programme: 104 (54%) patients who dropped out of the outpatient programme and 74 (50%) who dropped out of the detox programme lived in Abu Dhabi.
Readmission rates for 2013: The total number of admissions for 2013 was 1324. Of these admissions, 692 (52%) were readmitted one or more times to different programmes: 247 (36%) were readmitted to detox and 244 (35%) to outpatients, and 39 (6%) were readmitted involuntarily and 136 (20%) voluntarily.
Participation in continuing care
Data on the number of patients attending the continuing care groups or the medication appointments were not available in the electronic data system in 2013. Data obtained from the counsellor who runs the groups showed that 4–7 patients on average attended the evening groups on a weekly basis. Data from the outpatient clinic showed that 24 patients on average attended their long-term medication appointments. At the end of 2013, the NRC signed a memorandum of understanding with the department of labour and 3 patients were referred for employment after they had completed their treatment. There were also 66 patients who received financial assistance to pay loans and other bills while in treatment in 2013.
Patient satisfaction
At the end of treatment and prior to discharge, patients were asked to answer a survey regarding their experience at the NRC (Figure 4). The survey covered satisfaction with the many service components, including accessibility, admission process, facilities, treatment team. They were asked to rate each item on a scale of 0–100%. The results were very favourable and averaged over 75% satisfaction.
Discussion
This exercise was valuable as the first step to establishing clear criteria to measure the NRC programme performance. The findings were useful for programme performance and also served to highlight gaps for the organization. The team has come up with specific recommendations that can be grouped in three areas: operational issues, quality issues and electronic data system issues.
Operational issues
The number of no-shows was high. If all patients showed up for their scheduled appointments, it is not clear if the staff would be able to handle the work load. A clear policy on how to deal with no-shows is advised, e.g. overbooking, or not offering new appointments for repeated no-shows > 3 times.
A large number of families did not attend the recommended family visits. A clear directive is needed for the treatment team to invest in motivating families to attend these sessions. Offering family groups and home visits by the social workers for special cases may help.
The reasons why patients drop out need to be examined. For example, patients who travel long distances and others who have to return to their work sooner could benefit from a shorter course of intensive treatment or from linking them with a primary care doctor near their place of residence with consultation from an NRC professional. New and practical treatment interventions that take advantage of technology are highly recommended for the NRC to start as pilot programmes, such as tele-psychiatry (8), Internet-based therapy (9,10) and e-health.
Quality issues
Treatments programmes need to be documented as such with enrolment, retention and drop-out data recorded for each.
Medications need to be listed per patient.
The readmission rate in 2013 was 52%. Readmission rates to the detox and outpatient programmes were almost identical to the drop-out rates. The programme needs to invest in these areas to remedy the underlying reasons for drop-out and readmission.
More coordination is needed between services within the NRC as well as between other organizations for patient referral.
Electronic data system issues
- Data categorization: multiple subcategories that overlap and not clearly defined, should be streamlined and better defined.
- Data entry: grouping and better clarification of the different behavioural programmes is needed (e.g. contingency management, motivational interviewing, cognitive behavioural therapy), and the medications prescribed for treatment per patient need to be better recorded.
- Operational issues: standard operating procedures are needed for data extraction where staff roles and responsibilities are well defined.
Our objectives and the amount and type of data we wanted to capture may have been too ambitious for the newly operational electronic data system. There were more missing data than we anticipated. Further programme evaluations are expected to be smoother and shorter.
Conclusions
Evaluating the performance of addiction treatment programmes is crucial in this age of performance-based budget allocation and competing service organizations. We believe this exercise will improve the quality of services in our facility for the coming years. We hope that by sharing our experience other countries in the Middle East and North Africa region can benefit from it. As far as we know, there are no published data from any country in the Middle East and North Africa using similar programme performance methods. This will make it hard to compare our scale with others in the region. However, the items examined are straightforward and in line with international standards, and other countries in the region should find them easy to adopt, e.g. capacity and waiting time, drop-out rates and relapse. Furthermore, identifying the reasons for relapse and finding solutions for them will help improve treatment outcomes. Dealing with the stigma attached to drug addiction and mental illness and getting families involved (11,12) are common problems in the region. Almost three-quarters of the families of patients at our centre did not attend any family sessions. Family education and public awareness are crucial areas to focus on in order to improve addiction services in the Middle East and North Africa region.
Funding: None.
Competing interests: None declared.
References
- Sheedy CK, Whitter M. Guiding principles and elements of recovery-oriented systems of care: what do we know from the research? Rockville, MD: Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration; 2009.
- The treatment outcomes profile (TOP). Guidance notes for the reporting of TOP to NDTMS. National Treatment Agency for Substance Abuse; 2007 (http://www.nta.nhs.uk/uploads/top_ndtms_guidance_notes_190607.pdf, accessed 15 March 2017).
- Carise D, McLellan AT, Gifford LS. Development of a “Treatment Program” descriptor: The Addiction Treatment Inventory. Subst Use Misuse. 2000;35(12-14):1797–818.
- Garnik DW, Horgan CM, Acevedo A, McCorry F, Weisner C. Performance measures for substance abuse disorders-what research is needed? Addict Sci Clin Pract. 2012;7:18.
- Liddle HA, Dakof GA. Efficacy of family therapy for drug abuse: Promising but not definitive. J Marital Fam Ther. 1995;21(4):511–43.
- O’Farrell TJ, Fals-Stewart W. Behavioral couples therapy for alcoholism and drug abuse. J Subst Abuse Treat. 2000;18(1):51–4.
- Shapiro C. Integrating family-focused interventions into the criminal justice system. New York: The Vera Institute of Justice; 1999 (http://archive.vera.org/sites/default/files/resources/downloads/bodega_family.pdfaccessed 15 March 2017).
- Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: A 2013 review. Telemed J E Health. 2013;19(6):444-54.
- Barak A, Hen L, Boniel-Nissim M, Na’ama S. A comprehensive review and a meta-analysis of the effectiveness of Internet-based psychotherapeutic interventions. J Technol Hum Serv. 2008;26(2/4) doi.org/10.1080/15228830802094429.
- Gustafson DH, Boyle MG, Shaw BR, Isham A, McTavish F, Richards S, et al. An ehealth solution for people with alcohol problems. Alcohol Res Health. 2011;33(4):327-37
- Buchman DZ, Reiner P. Stigma and addiction: being and becoming. Am J Bioeth. 2009;9(9):18-9.
- Livingston JD, Milne T, Fang ML, Amari E. The effectiveness of interventions for reducing stigma related to substance use disorders: a systematic review. Addiction. 2012;107(1):39-50.
