Going to university is a time of great change for most students, and can exacerbate mental health struggles, as well as introduce new ones. More than 1 in 5 university students experience psychological distress, such as symptoms of depression and anxiety (Auerbach et al., 2016). Meeting academic expectations, coping with separation from a support network of family and home friends, and struggling to access appropriate support services (e.g. for disabilities) in a new setting, are just some of the things that are thought to impact upon university students’ mental health (Pedrelli et al., 2015).
In the past, if a student wanted support separate to clinical assistance (e.g. a therapy service), they might have reached out for a self-help book or video. Today, in addition to these, a person might find it even easier to access self-help support through their phone. A quick scan of your App store unearths all sorts of apps purporting to support a person with their mental health. But given the sheer variety of issues targeted by such apps, what kind of app-based support should you go for?
Newby et al. (2025) aimed to compare the effectiveness of 3 brief app-based support interventions targeted at Australian college students separated into categories of distress severity – mild, moderate and severe – to see which intervention(s) might prove most helpful depending on a student’s distress levels.
A quick scan of your App store unearths all sorts of apps purporting to support a person with their mental health.
Methods
The study enrolled and randomly allocated 1,394 students to one of 3 interventions or a control condition (twice-daily assessment of mood), and analysed data from 1,282 who completed the post-intervention assessment. The trial incorporated 12 mini-trials over a period of three years. Every student completed a 2 week socialisation to the app prior to randomisation, in which they completed twice-daily surveys about their mood. The researchers then used an AI-enhanced algorithm to assign students to the different interventions:
- Physical Activity, in which students set a physical activity goal as well as being given access to a training video
- Sleep Hygiene, which provided students with infographics designed to help them get into better sleeping habits
- Mindfulness, which provided guided audio meditations in addition to instructions for enhancing mindfulness during everyday activities
A further group of participants had to do twice-daily surveys of their mood through the same app, but none of the interventions. This was to provide an active control group for the 3 interventions to be compared against.
Randomisation in the first mini-trial allocated students to each of the groups on a 1:1:1:1 ratio. Data from this and each subsequent mini-trial was then fed into the algorithm as the study progressed to assign participants to the intervention deemed likeliest to give them the best outcome.
The researchers planned to continue the trial either until a significant difference was found between the most effective and the second-most effective intervention for each distress group, or after a maximum of 12 minitrials.
The primary outcome for the study was the change in self-rated psychological distress scores using the Depression Anxiety Stress Scale (DASS-21) before and after the intervention.
Adaptive trial design results in unequal probabilities of allocation across interventions. Multiple comparisons and repeated interim analyses throughout the trial also increase the risk of Type 1 errors (i.e. false positives). The researchers took this into account in their statistical analysis for the four groups, reweighting them so that marginal intervention effects could be detected as if all groups had been equally sampled.
Participants were randomised equally to one of three interventions or to a fourth control condition.
Results
The study had different findings on intervention effectiveness depending on distress severity. To account for the many comparisons being made at each time point, the authors used a statistical method called the Benjamini–Hochberg (BH) procedure, which adaptively adjusts significance thresholds across comparisons. This approach increases sensitivity to real effects while limiting the proportion of false positive findings.
Students with mild distress
Compared to the control group:
- Physical activity was significantly more effective in relieving distress: P = .007, SMD = 0.58 [95% CI, 0.30 to 0.86].
- Sleep hygiene was also significantly more effective: P = .01, SMD = 0.47 [95% CI, 0.20 to 0.73].
- Mindfulness was not significantly more effective: P = .03*, SMD = 0.34 [95% CI, 0.01 to 0.67].
Students with moderate distress
Compared to the control group:
- Physical activity was not significantly more effective in relieving distress: P = .02*, SMD = 0.45 [95% CI, 0.02 to 0.88].
- Sleep hygiene was also not significantly more effective: P = .02*, SMD = 0.39 [95% CI, 0.01 to 0.76].
- Mindfulness was also not significantly more effective: P = .008*, SMD = 0.47 [95% CI, 0.09 to 0.84].
Students with severe distress
Compared to the control group:
- Physical activity was significantly more effective in relieving distress: P = .01, SMD = 0.62 [95% CI, 0.23 to 1.02].
- Sleep hygiene was not significantly more effective: P = .04*, SMD = 0.12 [95% CI, −0.26 to 0.50].
- Mindfulness was significantly more effective in relieving distress: P = .03, SMD = 0.53 [95% CI, 0.19 to 0.87].
* These BH-adjusted p-values were not lower than their BH-adjusted critical P value.
Comparisons between interventions were largely non-significant, with the exception of sleep hygiene being significantly less effective than either mindfulness (P = .02, SMD −0.41 [−0.69 to −0.13]) or physical activity (P = .07, SMD = -0.50 [-0.16 to -0.84]),
The findings indicate that there may be differences in effectiveness between different kinds of support depending on a student’s initial level of distress.
Physical activity has long been associated with increased mental wellbeing and quality of life (e.g. Mahindru et al., 2023), with much evidence pointing to its utility in counteracting some of the symptoms of anxiety and depression (Wegner et al., 2014). In this study, app-based support for physical activity appeared to have significant moderate benefits for those with either mild or severe distress.
That sleep hygiene had a significant moderate effect for those with mild distress, but not others, suggests that those experiencing higher levels of distress may need more intensive clinical support. Similarly, findings in this study regarding the moderate effectiveness of app-based mindfulness support for those experiencing severe distress are similar to those of other randomised controlled trials (Gál et al., 2021).
The findings indicate that there may be differences in effectiveness between different kinds of support depending on a student’s initial level of distress.
Conclusions
The potential disparity in effectiveness of the app-based interventions depending on distress severity indicates a need for further research into app-based interventions tailored to a person’s distress levels.
Physical activity seemed to be effective for people with either mild or severe distress, while sleep hygiene was useful for mild distress, while mindfulness proved more useful for people with severe distress. For severe distress, support for sleep hygiene seems to be significantly less effective than either support for physical activity or mindfulness.
The findings demonstrate potential for treatment personalisation in research (rather than fixed allocation randomised controlled trials) at scale. Further research could compare the efficiency and utility of AI-enhanced response-adaptive trials with conventional RCTs in mental health.
Strengths and limitations
Strengths
The study used an ambitious design to assess the effectiveness of different app-based interventions for students with different levels of distress. Although the numerous comparisons possibly reduced its statistical power to detect small to moderate benefits, it nevertheless recruited a substantial number of students, and the adaptive design aimed to allocate students to more efficient interventions, often considered a strength of these studies from an ethical perspective.
One particular strength of the trial is the low rate of missing data at the post-intervention assessments (8%). The use of a 2-week period of twice-daily assessments of mood prior to randomisation may have helped with this. The authors don’t report how many participants withdrew during this ‘onboarding’ period. However, it seems likely that those who persisted with this may have been more likely to persist with the subsequent 2 weeks of intervention and assessment.
The study design is clearly described in the protocol, and the procedure followed throughout the study is faithful to the description in the protocol, including outcome measures to be used. The protocol and supplementary material appear sufficiently detailed for the study to be replicable (the code for the algorithm used in the study design has been offered upon request). Furthermore, the study used validated measures, and the implications of the adaptive trial design were taken into careful consideration in the statistical analysis, reweighting the different groups to take into account unequal distribution of participants across the interventions and the control group.
Limitations
The study has some limitations that suggest the need for further research. Firstly, the interventions delivered to students were brief, lasting only two weeks, and the minimum engagement was met by only two-thirds of participants, with only 29.3% of participants accessing all content from the app.
Secondly, there is a risk that even within student populations, that the findings are not generalisable, as well over 70% of participants were female Australian domestic students, with only small representation of Aboriginal and Indigenous students. Primary language was reported on by the study, but not race or ethnicity, making it difficult to ascertain the racial diversity of the sample.
Thirdly, the characteristics fed into the algorithm were based only on self-reported measures of distress: other characteristics which might usefully inform personalised, effective treatments, such as socio-demographics, or specific university-based stressors, could be fed into such an algorithm in further research. The mini-trial was not included as a covariate, although participant characteristics may well have changed over time in each group across the twelve mini-trials. Interactions between engagement and compliance, self-reported life events collected post-intervention and intervention compliance, on which data was collected, are to be explored in a future paper.
The clinical significance of the between-group differences is a little unclear. We found only one study reporting a ‘minimal clinically important difference’ (MCID) in DASS-21 scores (Yohannes et al., 2019). This reported MCIDs ranging from 3.6 (anxiety) to 7.2 points (stress), however this was a study of elderly people (mean age 71.6) with chronic obstructive pulmonary disease, not students experiencing mild to severe distress. Precise mean differences for the current trial are not reported, but we can roughly estimate them by multiplying the SMD by the baseline SD. If we use the smallest (11.8) and largest (20) baseline SDs, then the plausible post-treatment mean differences for the effects of sleep hygiene and physical activity on those with initially mild distress range from 5.5 to 11.6. For those with initially severe distress, the plausible post-treatment mean differences attributable to mindfulness or physical activity range from 6.3 to 12.4. For such a widely used measure, the lack of a clear MCID or minimally important difference (MID) seems to be a gap in the evidence-base. If we don’t know what mean differences in DASS-21 scores signify, then we are left to rely on the SMD and Cohen’s heuristic criteria for interpreting it (small = 0.2, moderate = 0.5, large = 0.8).
Although the overall sample size was large, many comparisons were performed. The authors adjusted for this, but this adjustment may have reduced their ability to detect smaller yet potentially important effects. Non-significant estimates were generally quite imprecise. This may be because the trial had reduced statistical power to detect smaller yet potentially meaningful effects. For example, although the effect of physical activity on those with moderate distress was not significant, null through to large effects were included within the 95% confidence intervals.
Finally, there appeared to be no inclusion of a measure of adverse events or experiences caused by the intervention. Due to the general lack of regulation or oversight of mental health support apps for end-users, particularly given that some in this study self-reported as experiencing moderate or severe distress, some measure of adverse experiences could provide further useful information about the effectiveness and potential risks of app-based interventions.
Self report measures may reduce the reliability of results.
Implications for practice
The study provides useful recommendations for the direction of further research. Much is still unknown about the usefulness and safety of app-based interventions for mental health support. The study contributes to a growing body of research into the effectiveness of smartphone apps in supporting people with their mental health, as apps are increasingly available worldwide offering a variety of support options and interventions (Torous et al., 2025). Very few apps are regulated, and the assumption that apps are helpful – and that their interventions are as useful as one another – should not go unexamined.
This study highlights that different interventions, even brief interventions, might be more helpful than others depending on the particular characteristics of the person. Further research could focus on a variety of characteristics that might support the effectiveness and personalisation of app-based support.
The study also indicates that for those with moderate or severe distress, clinical support may be necessary in addition to, or instead of, app-based interventions. Future research could focus on whether, and how, app-based interventions can be used as an evidence-based adjunct to clinical support for mental health struggles. Furthermore, future research into the efficacy of digital interventions could consider the potential benefits of adaptive trials and tailored allocation in their study design, provided that appropriate statistical methods are used to reduce the likelihood of a Type-1 (false positive) error.
The study demonstrates the potential benefits of adaptive trials.
Statement of interests
James Martin and Paul Hutton report no conflicts of interest.
Edited by
Dr Simon Bradstreet.
Links
Primary paper
Newby, J., Gupta, S., Hoon, L., Zheng, W., Whitton, A. E., Huckvale, K., … & Christensen, H. (2025). Brief Digital Interventions for Psychological Distress: An AI-Enhanced Response-Adaptive Randomized Clinical Trial. JAMA Network Open, 8(10), e2540502.
Other references
Auerbach, R. P., Alonso, J., Axinn, W. G., Cuijpers, P., Ebert, D. D., Green, J. G., … Bruffaerts, R. (2016). Mental disorders among college students in the World Health Organization World Mental Health Surveys. Psychological Medicine, 46(14), 2955–2970.
Gál, É., Ștefan, S., & Cristea, I. A. (2021). The efficacy of mindfulness meditation apps in enhancing users’ well-being and mental health related outcomes: a meta-analysis of randomized controlled trials. Journal of affective disorders, 279, 131-142.
Mahindru, A., Patil, P., & Agrawal, V. (2023). Role of physical activity on mental health and well-being: A review. Cureus, 15(1).
Pedrelli, P., Nyer, M., Yeung, A., Zulauf, C., & Wilens, T. (2015). College students: mental health problems and treatment considerations. Academic psychiatry, 39(5), 503-511.
Torous, J., Linardon, J., Goldberg, S. B., Sun, S., Bell, I., Nicholas, J., … & Firth, J. (2025). The evolving field of digital mental health: current evidence and implementation issues for smartphone apps, generative artificial intelligence, and virtual reality. World Psychiatry, 24(2), 156-174.
Wegner, M., Helmich, I., Machado, S., E Nardi, A., Arias-Carrion, O., & Budde, H. (2014). Effects of exercise on anxiety and depression disorders: review of meta-analyses and neurobiological mechanisms. CNS & Neurological Disorders-Drug Targets (Formerly Current Drug Targets-CNS & Neurological Disorders), 13(6), 1002-1014.
Yohannes, A. M., Dryden, S., & Hanania, N. A. (2019). Validity and responsiveness of the Depression Anxiety Stress Scales-21 (DASS-21) in COPD. Chest, 155(6), 1166-1177.
