- Developing a Sleep Algorithm to Support a Digital Medicine System: Noninterventional, Observational Sleep Study
Background: Sleep-wake patterns are important behavioral biomarkers for patients with serious mental illness (SMI), providing insight into their well-being. The gold standard for monitoring sleep is polysomnography (PSG), which requires a sleep lab facility; however, advances in wearable sensor technology allow for real-world sleep-wake monitoring. Objective: The goal of this study was to develop a PSG-validated sleep algorithm using accelerometer (ACC) and electrocardiogram (ECG) data from a wearable patch to accurately quantify sleep in a real-world setting. Methods: In this noninterventional, nonsignificant-risk, abbreviated investigational device exemption, single-site study, participants wore the reusable wearable sensor version 2 (RW2) patch. The RW2 patch is part of a digital medicine system (aripiprazole with sensor) designed to provide objective records of medication ingestion for patients with schizophrenia, bipolar I disorder, and major depressive disorder. This study developed a sleep algorithm from patch data and did not contain any study-related or digitized medication. Patch-acquired ACC and ECG data were compared against PSG data to build machine learning classification models to distinguish periods of wake from sleep. The PSG data provided sleep stage classifications at 30-second intervals, which were combined into 5-minute windows, and labeled as sleep or wake based on the majority of sleep stages within the window. ACC and ECG features were derived for each 5-minute window. The algorithm that most accurately predicted sleep parameters against PSG data was compared to commercially available wearable devices to further benchmark model performance. Results: Of 80 participants enrolled, 60 had at least 1 night of analyzable ACC and ECG data (25 healthy volunteers and 35 participants with diagnosed SMI). Overall, 10,574 5-minute valid windows were identified (5854 from participants with SMI) and 83% were classified as greater than half sleep. Of 3 models tested, the conditional random field (CRF) algorithm provided the most robust sleep-wake classification. Performance was comparable to the middle 50% of commercial devices evaluated in a recent publication, providing sleep detection performance of 0.93 (sensitivity) and wake detection performance of 0.60 (specificity) at a prediction probability threshold of 0.75. The CRF algorithm retained this performance for individual sleep parameters, including total sleep time, sleep efficiency, and wake after sleep onset (within the middle 50% to top 25% of the assessed devices). The only parameter where the model performance was lower was sleep onset latency (within the bottom 25% of all comparator devices). Conclusions: Using industry-best practices, we developed a sleep algorithm for use with the RW2 patch that can accurately detect sleep and wake windows compared to PSG-labeled sleep data. This algorithm may be used for a more complete understanding of well-being for patients with SMI in a real-world setting, without the need for PSG and a sleep lab.
- Early Attrition Prediction for Web-Based Interpretation Bias Modification to Reduce Anxious Thinking: A Machine Learning Study
Background: Digital mental health is a promising paradigm for individualized, patient-driven health care. For example, cognitive bias modification programs that target interpretation biases (cognitive bias modification for interpretation [CBM-I]) can provide practice thinking about ambiguous situations in less threatening ways on the web without requiring a therapist. However, digital mental health interventions, including CBM-I, are often plagued with lack of sustained engagement and high attrition rates. New attrition detection and mitigation strategies are needed to improve these interventions. Objective: This paper aims to identify participants at a high risk of dropout during the early stages of 3 web-based trials of multisession CBM-I and to investigate which self-reported and passively detected feature sets computed from the participants interacting with the intervention and assessments were most informative in making this prediction. Methods: The participants analyzed in this paper were community adults with traits such as anxiety or negative thinking about the future (Study 1: n=252, Study 2: n=326, Study 3: n=699) who had been assigned to CBM-I conditions in 3 efficacy-effectiveness trials on our team’s public research website. To identify participants at a high risk of dropout, we created 4 unique feature sets: self-reported baseline user characteristics (eg, demographics), self-reported user context and reactions to the program (eg, state affect), self-reported user clinical functioning (eg, mental health symptoms), and passively detected user behavior on the website (eg, time spent on a web page of CBM-I training exercises, time of day during which the exercises were completed, latency of completing the assessments, and type of device used). Then, we investigated the feature sets as potential predictors of which participants were at high risk of not starting the second training session of a given program using well-known machine learning algorithms. Results: The extreme gradient boosting algorithm performed the best and identified participants at high risk with macro–F1-scores of .832 (Study 1 with 146 features), .770 (Study 2 with 87 features), and .917 (Study 3 with 127 features). Features involving passive detection of user behavior contributed the most to the prediction relative to other features. The mean Gini importance scores for the passive features were as follows: .033 (95% CI .019-.047) in Study 1; .029 (95% CI .023-.035) in Study 2; and .045 (95% CI .039-.051) in Study 3. However, using all features extracted from a given study led to the best predictive performance. Conclusions: These results suggest that using passive indicators of user behavior, alongside self-reported measures, can improve the accuracy of prediction of participants at a high risk of dropout early during multisession CBM-I programs. Furthermore, our analyses highlight the challenge of generalizability in digital health intervention studies and the need for more personalized attrition prevention strategies.
- Implementing Findable, Accessible, Interoperable, Reusable (FAIR) Principles in Child and Adolescent Mental Health Research: Mixed Methods Approach
Background: The FAIR (Findable, Accessible, Interoperable, Reusable) data principles are a guideline to improve the reusability of data. However, properly implementing these principles is challenging due to a wide range of barriers. Objective: To further the field of FAIR data, this study aimed to systematically identify barriers regarding implementing the FAIR principles in the area of child and adolescent mental health research, define the most challenging barriers, and provide recommendations for these barriers. Methods: Three sources were used as input to identify barriers: 1) evaluation of the implementation process of the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) by three data managers, 2) interviews with experts on mental health research, reusable health data, and data quality, and 3) a rapid literature review. All barriers were categorized according to Type as described by Cabana et al. (1999), the affected FAIR principle, a category to add detail about the origin of the barrier, and whether a barrier was mental health specific. The barriers were assessed and ranked on impact with the data managers using the Delphi method. Results: Thirteen barriers were identified by the data managers, seven were identified by the experts, and 30 barriers were extracted from the literature. This resulted in 45 unique barriers. The characteristics that were most assigned to the barriers were respectively: external Type (n=32/45) (e.g., organizational policy preventing the use of required software), tooling Category (n=19/45) (i.e., software and databases), all FAIR principles (n=15/45), and not mental health specific (n=43/45). Consensus on ranking the scores of the barriers was reached after two rounds of the Delphi method. The most important recommendations to overcome the barriers are adding a FAIR data steward to the research team, accessible step-by-step guides, and ensuring sustainable funding for the implementation and long-term use of FAIR data. Conclusions: By systematically listing these barriers and providing recommendations we intend to enhance the awareness of researchers and grant providers that making data FAIR demands specific expertise, available tooling, and proper investments.
- Integrating Patient-Generated Digital Data Into Mental Health Therapy: Mixed Methods Analysis of User Experience
Background: Therapists and their patients increasingly discuss digital data from social media, smartphone sensors, and other online engagement within the context of psychotherapy. Objective: We examined patients’ and mental health therapists’ experiences and perceptions following a randomized controlled trial (RCT) in which they both received regular summaries of patients’ digital data (e.g., dashboard) to review and discuss in session. The dashboard included data which patients consented to share from their social media posts, phone usage and online searches. Methods: Following the RCT, patient (n=56) and therapist (n=44) participants completed a debriefing survey after their study completion (from December 2021 - January 2022). Participants were asked about their experience receiving a digital data dashboard in psychotherapy via closed- and open-ended questions. We calculated descriptive statistics for closed-ended questions and conducted qualitative coding via NVivo 10 and natural language processing using the machine learning tool Latent Dirichlet Allocation to analyze open-ended questions. Results: Of 100 participants, nearly half (49%) described their experience with the dashboard as “positive,” while the other half noted a “neutral” experience. Responses to the open-ended questions resulted in three thematic areas (9 sub-categories): (1) dashboard experience (positive; neutral or negative; comfortable), (2) perception of the dashboard’s impact on enhancing therapy (accountability; increased awareness over time; objectivity), and (3) dashboard refinements (additional sources; tailored content; ethics). Conclusions: Patients reported that receiving their digital data helped them stay “accountable,” while therapists indicated that dashboard helped “tailor treatment plans.” Patient and therapist surveys provided important feedback their experience regularly discussing dashboards in psychotherapy.
- Ecological Momentary Assessment of Self-Harm Thoughts and Behaviors: Systematic Review of Constructs From the Integrated Motivational-Volitional Model
Background: The integrated motivational-volitional model (IMV) is one of the leading theoretical models of suicidal thoughts and behavior. There has been a recent proliferation in the assessment of suicidal and nonsuicidal self-harm thoughts and behaviors (SHTBs) in daily life. Objective: This systematic review synthesized evidence from ecological momentary assessment (EMA) studies in the SHTB literature to address the following questions: (1) Which constructs in the IMV model have been assessed using EMA, and how have they been assessed? (2) Do different constructs from the IMV model fluctuate in daily life? (3) What is the relationship between the different IMV constructs and SHTBs in daily life? Methods: Consistent with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, we conducted systematic searches of 5 databases—Web of Science, Embase, MEDLINE, PsycINFO, and Europe PMC Preprints—from inception to March 26, 2024. Results: Our searches resulted in the inclusion and narrative synthesis of 53 studies across 58 papers. A total of 15 IMV constructs were measured using EMA across the included papers. The most frequently measured constructs were thwarted belongingness (24/58, 41% of the papers), future thinking (20/58, 34% of the papers), and perceived burdensomeness (16/58, 28% of the papers). The least frequently measured constructs were humiliation, social problem-solving, mental imagery, and perceived capability for suicide. None of the included papers measured memory biases, goals, norms, or resilience using EMA. Comparison of intraclass correlation coefficients (45/58, 78% of the papers) revealed moderate but inconsistent within-person variance across all the examined constructs. We found evidence (39/58, 67% of the papers) of concurrent associations between almost all constructs and SHTBs in daily life, with some evidence that entrapment, shame, rumination, thwarted belongingness, hopelessness, social support, and impulsivity are additionally associated with SHTBs in lagged (ie, longitudinal) relationships. Conclusions: Comparisons were hindered by variation in methodology, including the populations studied, EMA sampling scheme, operationalization of IMV constructs and SHTBs, and statistical approach used. Our findings suggest that EMA studies are a useful methodology for examining risk factors for SHTBs; however, more research is needed for some IMV constructs. Quality assessment suggested several areas for improvement in the reporting of EMA studies in this field. Trial Registration: PROSPERO CRD42022349514; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=349514
- Text Messaging to Extend School-Based Suicide Prevention: Pilot Randomized Controlled Trial
Background: Suicide is the third-leading cause of death among US adolescents aged 10-19 years, and about 10% attempt suicide each year. School-based universal prevention may reduce youth suicidal behavior. Sources of Strength uses a peer leader network diffusion model to promote healthy norms across a school population. A key challenge within schoolwide programs is reaching a large and diverse array of students, especially those less engaged with their peers. Motivated by this challenge, we developed and field-tested Text4Strength—a program of automated text messages targeting help-seeking attitudes and norms, social coping resources, and emotion regulation skills. Objective: This study conducted a pilot randomized controlled trial of Text4Strength in 1 high school as an extension of an ongoing schoolwide program (Sources of Strength), to test its impact on targets that have the potential to reduce suicidal behavior. Methods: Students at an upstate New York high school (N=223) received 1-2 text messages per week for 9 weeks, targeting strategies for coping with difficult feelings and experiences through clarifying emotions and focusing on positive affect concepts, awareness, and strengthening of youth-adult relationships; and positive help-seeking norms, skills, and resources. Surveys were administered at baseline, immediately post intervention and 3 months after texting ended. We measured proximal intervention targets (methods of coping during stressful events, ability to make sense of their own emotions, feelings of powerlessness during emotion management and recovery, relations with trusted adults at school, and help-seeking behaviors), symptoms and suicide ideation, and student replies to messages. Results: No significant effects were observed for any outcome at either follow-up time point. Results showed that if there is a true (but undetected) intervention effect, it is small. Students with fewer friend nominations did not interact any more or less with the text messages. Exploratory moderation analyses observed no interaction between the intervention condition and the number of friends or baseline suicide ideation at any time point. Conclusions: In contrast to a promising previous field test, these results suggest that Text4Strength is unlikely to have impacted the outcomes of interest and that undetected moderate or large effects can be ruled out with high confidence. Although motivated by the need to reach more isolated students, students with fewer friends did not engage more or show a greater effect than other participants. This study was conducted in a single high school that was already implementing Sources of Strength, so the bar for showing a distinct effect from texting alone was high. Many further channels for reaching youth through private messaging remain unexplored. Alternative delivery systems should be investigated, such as embedding messaging in gaming chat systems and other media. More sophisticated systems drawing on chatbots may also achieve better outcomes. Trial Registration: ClinicalTrials.gov NCT03145363; https://clinicaltrials.gov/study/NCT03145363
- The Use of a Digital Well-Being App (Stay Strong App) With Indigenous People in Prison: Randomized Controlled Trial
Background: Indigenous Australians in custody experience much greater rates of poor mental health and well-being than those of the general community, and these problems are not adequately addressed. Digital mental health strategies offer innovative opportunities to address the problems, but little is known about their feasibility in or impact on this population. Objective: This study aims to conduct a pilot trial evaluating the impact of adding the Stay Strong app to mental health and well-being services for Indigenous women and men in custody. The trial compared immediate and 3-month delayed use of the app by the health service, assessing its effects on well-being, empowerment, and psychological distress at 3 and 6 months after the baseline. Methods: Indigenous participants were recruited from 3 high-security Australian prisons from January 2017 to September 2019. The outcome measures assessed well-being (Warwick-Edinburgh Mental Wellbeing Scale), empowerment (Growth and Empowerment Measure [GEM]—giving total, 14-item Emotional Empowerment Scale, and 12 Scenarios scores), and psychological distress (Kessler Psychological Distress Scale). Intention-to-treat effects on these outcomes were analyzed using linear mixed models. Results: Substantial challenges in obtaining ethical and institutional approval for the trial were encountered, as were difficulties in timely recruitment and retention due to staff shortages and the release of participants from prison before follow-up assessments and an inability to follow up with participants after release. A total of 132 prisoners (age: mean 33, SD 8 y) were randomized into either an immediate (n=82) or a delayed treatment (n=52) group. However, only 56 (42.4%) could be assessed at 3 months and 37 (28%) at 6 months, raising questions concerning the representativeness of the results. Linear improvements over time were seen in all outcomes (GEM total: Cohen d=0.99; GEM 14-item Emotional Empowerment Scale: Cohen d=0.94; GEM 12 Scenarios: Cohen d=0.87; Warwick-Edinburgh Mental Wellbeing Scale: Cohen d=0.76; Kessler Psychological Distress Scale: Cohen d=0.49), but no differential effects for group or the addition of the Stay Strong app were found. Conclusions: We believe this to be Australia’s first evaluation of a digital mental health app in prison and the first among Indigenous people in custody. While the study demonstrated that the use of a well-being app within a prison was feasible, staff shortages led to delayed recruitment and a consequent low retention, and significant beneficial effects of the app’s use within a forensic mental health service were not seen. Additional staff resources and a longer intervention may be needed to allow a demonstration of satisfactory retention and impact in future research. Trial Registration: ANZCTR ACTRN12624001261505; https://www.anzctr.org.au/ACTRN12624001261505.aspx
- Digital Health Interventions for Informal Family Caregivers of People With First-Episode Psychosis: Systematic Review on User Experience and Effectiveness
Background: First-episode psychosis (FEP) imposes a substantial burden not only on the individual affected but also on their families. Given that FEP usually occurs during adolescence, families overtake a big part of informal care. Early family interventions, especially psychoeducation, are crucial for informal family caregivers to best support the recovery of their loved one with FEP and to reduce the risk of a psychotic relapse as much as possible, but also to avoid chronic stress within the family due to the burden of care. Digital health interventions offer the possibility to access help quicker, use less resources, and improve informal family caregiver outcomes, for example, by reducing stress and improving caregiver quality of life. Objective: This study aimed to systematically identify studies on digital health interventions for informal family caregivers of people with FEP and to describe and synthesize the available literature on user experience, as well as the effectiveness of such digital applications on the clinical outcomes, consisting of (1) perceived caregiver stress, (2) expressed emotion, and (3) parental self-efficacy. Methods: A systematic search was carried out across 4 electronic databases. In addition, reference lists of relevant studies were hand-searched. This review aimed to include only primary studies on informal family caregivers, who had to care for a person with FEP between 15 years and 40 years of age and a diagnosis of FEP with onset of observed symptoms within the past 5 years. All types of digital interventions were included. This systematic review is aligned with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) 2020 guidelines. Results: The search identified 7 studies that reported on user experience or effectiveness of digital health interventions on perceived caregiver stress, expressed emotion, and parental self-efficacy, including 377 informal family FEP caregivers across trials. Digital health interventions–web-based, videoconferences, and mHealth–were well accepted and perceived as relevant, easy to use, and helpful by informal family FEP caregivers. Psychoeducational content was rated as the most important across studies. Perceived caregiver stress, expressed emotion, and parental self-efficacy improved in all studies that reported on these clinical outcomes. Conclusions: The results of this review suggest that digital health interventions aimed at informal family caregivers of individuals with FEP can improve relevant clinical outcomes, with participants reporting a positive user experience. However, for some interventions reviewed, specialized in-person family care outperformed the digital intervention and partially led to better results in perceived caregiver stress and parental self-efficacy. Therefore, while digital interventions present a promising approach to alleviate the burden of care and improve informal family FEP caregiver outcomes, more studies with well-powered experimental designs are needed to further investigate the effectiveness of such applications in this population. Clinical Trial: PROSPERO CRD42024536715; https://tinyurl.com/bdd3u7v9
- Avatar Intervention in Virtual Reality for Cannabis Use Disorder in Individuals With Severe Mental Disorders: Results From a 1-Year, Single-Arm Clinical Trial
Background: The dual diagnosis of cannabis use disorder (CUD) and severe mental disorder (SMD) results in clinically complex individuals. Cannabis use is known to have negative consequences on psychiatric symptoms, medication compliance, and disease prognosis. Moreover, the effectiveness of currently available psychotherapeutic treatments is limited in this population. Objective: In this context, our research team developed avatar intervention, an approach using virtual reality as a therapeutic tool to treat CUD in individuals with SMD. Methods: Over the course of the 8 intervention sessions, participants were given the opportunity to enter a dialogue with an avatar representing a person with a significant role in their consumption, who was animated in real time by a therapist. The primary outcomes were the quantity of cannabis consumed and the frequency of use. Secondary outcomes included severity of problematic cannabis use, motivation for change, protective strategies for cannabis use, consequences of cannabis use, psychiatric symptoms, and quality of life. Changes in reported outcomes during the assessment periods before the intervention, post intervention and 3-, 6 and 12 months after the end of the intervention were assessed using a linear mixed-effects model. Results: Significant reductions were observed in the quantity of cannabis consumed, and these were maintained until the 12-month follow-up visit (d =0.804, p<0.001; confirmed by urine quantification). Frequency of cannabis use showed a small significant reduction at the 3-month follow-up (d=0.384, p=0.031). Moreover, improvements were observed for severity of CUD, cannabis related negative consequences, motivation to change cannabis use and in the strategies employed to mitigate harms related to cannabis use. Finally, moderate benefits were observed for quality of life and psychiatric symptoms. Conclusions: Overall, this unique intervention shows promising results that seem to be maintained up to 12 months after the end of the intervention. With the aim of overcoming the methodological limitations of a pilot study, a single-blind randomized controlled trial is currently underway to compare the avatar intervention for CUD with a conventional addiction intervention. Clinical Trial: ClinicalTrials.gov NCT05726617; https://clinicaltrials.gov/study/NCT05726617
- Effects of a Digital Therapeutic Adjunct to Eating Disorder Treatment on Health Care Service Utilization and Clinical Outcomes: Retrospective Observational Study Using Electronic Health Records
Background: The need for scalable solutions facilitating access to eating disorder (ED) treatment services that are efficient, effective, and inclusive is a major public health priority. Remote access to synchronous and asynchronous support delivered via health apps has shown promise, but results are so far mixed, and there are limited data on whether apps can enhance health care utilization. Objective: This study aims to examine the effects of app-augmented treatment on clinical outcomes and health care utilization for patients receiving treatment for an ED in outpatient and intensive outpatient levels of care. Methods: Recovery Record was implemented in outpatient and intensive outpatient services in a California-based health maintenance organization. We examined outcomes for eligible patients with ED by comparing clinical and service utilization medical record data over a 6-month period after implementation with analogous data for the control group in the year prior. We used a logistic regression model and inverse-weighted estimates of the probability of treatment to adjust for treatment selection bias. Results: App-augmented treatment was associated with a significant decrease in emergency department visits (P<.001) and a significant increase in outpatient treatment utilization (P<.001). There was a significantly larger weight gain for patients in low-weight categories (ie, underweight, those with anorexia, or those with severe anorexia) with app-augmented treatment (treatment effect: 0.74, 0.25, and 0.35, respectively; P=.02), with a greater percentage of patients moving into a higher BMI class (P=.01). Conclusions: Integrating remote patient engagement apps into ED treatment plans can have beneficial effects on both clinical outcomes and service utilization. More research should be undertaken on long-term efficacy and cost-effectiveness to further explore the impact of digital health interventions in ED care.