Safe Enough? Building Trust in AI for Mental Health

The use of AI chatbots as a tool for mental health support is not a speculative future, but already the reality for one in four people in the UK who may otherwise struggle to access timely mental health care. This present reality, where AI has already grown into a significant part of the mental health landscape with little oversight or regulation, is where our panel discussions and network debates were situated. We asked attendees and our panels of clinicians, innovators, and regulatory experts to consider what ‘safe’ and ‘evidence based’ actually means in the context of AI, and how the UK can create conditions that are conducive to responsible and accountable innovation in this area.

Our clinical panel, moderated by Dr Jenny Shand who was joined by Dr Anna Mandeville, Alastair Santhouse, and Fran Zanatta, drew together successful administrative use-cases alongside demonstrable issues around AI sycophancy and safeguarding failures to illustrate the wide range of potential applications and dangers of AI in a therapeutic context. These perspectives culminated in a discussion of tensions between tools designed to maximise shareholder profit and the need for care that is always first and foremost patient-centred; grounding the day’s conversations in the most important issue, the safety and wellbeing of those seeking mental health support.

In our second panel discussion turned to how to design tools built around Audrey Tang’s principle of “AI in the human loop”, a philosophy from the Digital Minister of Taiwan which argues that AI tools should be built to fit into human infrastructure allowing for robust and consistent human oversight. Samuel Stern, Ross Harper, and Mark Ungless (from Affiniti AI, Limbic AI, and Slingshot AI respectively) led a fascinating discussion on how to build tools and businesses that don’t necessitate maximising user engagement to operate successfully, but instead are centred around social impact and positive mental health outcomes.

Finally Alix Toffin, Dr Aislinn Gomez-Bergin, and Dr Ed Millgate, brought a systems perspective to the table, highlighting present issues around regulation, evidence generation, and equity. With minimum safety standards for these tools still undefined, they highlighted the urgent need to create processes through which we can continually evaluate and safeguard the use of AI within the mental health ecosystem. The need for improved focus on the ways in which these tools may re-entrench health inequalities while appearing to improve access provided a thought provoking start to the open discussion between panelists and attendees which drew together insights from the morning.

At the end of the day stakeholders, innovators, clinicians, regulators, and people with lived experience worked to develop shared, practice-level understandings of the risks and potential avenues for safeguarding within our current context of AI use by millions seeking mental health support.

MHIN will continue to facilitate these conversations. Our next event is the Mental Health Innovation Showcase in partnership with Harley Street Health District on 15 September, focusing on mental health solutions navigating the UK landscape in holistic ways. Stay tuned for updates from MHIN and HSHD regarding registration and participation in the showcase.