DioSim
Summary
DioSim enables one to understand the time and energy involved in achieving a desired outcome via a two-party dialogue. Linked with multi-agent dialogue analysis, it can also be used to model potential outcomes between human-to-human and Human-to-AI Agent interactions.
DioSim was tested in a case study below. The aim was to see if it could model the time and energy required to achieve positive behavioural changes in therapeutic relationships based solely on analysis of AI agents talking to each other. The aim was to validate DioSIm's value in modelling caseload management. The motivation behind the research was to understand ways in which one could prevent therapist burnout while maintaining therapeutic effectiveness.
Results
Below is an output of what DioSim believed is the process associated with the delivery of CBT therapies
DioSim enables one to understand the time and energy involved in achieving a desired outcome via a two-party dialogue. Linked with multi-agent dialogue analysis, it can also be used to model potential outcomes between human-to-human and Human-to-AI Agent interactions.
DioSim was tested in a case study below. The aim was to see if it could model the time and energy required to achieve positive behavioural changes in therapeutic relationships based solely on analysis of AI agents talking to each other. The aim was to validate DioSIm's value in modelling caseload management. The motivation behind the research was to understand ways in which one could prevent therapist burnout while maintaining therapeutic effectiveness.
Results
Below is an output of what DioSim believed is the process associated with the delivery of CBT therapies
The above was validated by NHS clinicians as a fair representation of what should happen in the NHS if a clinician can access a patient.
Method
CarefulAI researchers used DioSim to create detailed personas with specific psychological backgrounds, trauma histories, therapeutic needs, and clinicians. These were embodied in multiple dialogue agents, powered by LLMs, and trained using detailed prompts. The agents were enabled to talk with each other using an open-source multi-agent system (TinyFactory). The dialogues were logged, and insights were generated using pre-trained LLMs.
Discussion
AI agents set to the task of simulating human interaction can apparently derive the current state of the human-to-human system more quickly than hitherto possible. This multi-agent AI system approach allowed for the simulation of group therapy dynamics, family therapy sessions, and multidisciplinary team interactions. The approach generated detailed session transcripts and outcome measures, enabling:
- Analysis of therapeutic techniques
- Evaluation of intervention effectiveness
- Identification of training needs
- Development of best practices
- Research into therapeutic processes
The cost-effectiveness of the approach tool for therapy training, research, and service development, offering risk-free experimentation and learning opportunities, became transparent during its implementation. Human-to-human alternatives would take years and millions of pounds to gather data and years to analyse. The above output was achieved in 7 days.
The approach's future strengths may lay in its capability to simulate other environments
1. Complex therapeutic interventions
2. Explore challenging clinical scenarios e.g. Human-to-AI Agent
3. Test different therapeutic approaches
4. Develop and refine treatment protocols
5. Train new therapists in a safe environment
Future Work
Therapists involved in the work concluded that there is some value in forming a community of practice to explore approaches like DioSim. To this end, CarefulAI has committed to support its development.
The steps associated with the practice development are understanding the value of using multi-agent modelling systems like TinyTroupe to explore decreasing costs in the NHS.
Specifically, the time and expense of people outside CarefulAI setting up such simulations and doing the same analysis.
Method
CarefulAI researchers used DioSim to create detailed personas with specific psychological backgrounds, trauma histories, therapeutic needs, and clinicians. These were embodied in multiple dialogue agents, powered by LLMs, and trained using detailed prompts. The agents were enabled to talk with each other using an open-source multi-agent system (TinyFactory). The dialogues were logged, and insights were generated using pre-trained LLMs.
Discussion
AI agents set to the task of simulating human interaction can apparently derive the current state of the human-to-human system more quickly than hitherto possible. This multi-agent AI system approach allowed for the simulation of group therapy dynamics, family therapy sessions, and multidisciplinary team interactions. The approach generated detailed session transcripts and outcome measures, enabling:
- Analysis of therapeutic techniques
- Evaluation of intervention effectiveness
- Identification of training needs
- Development of best practices
- Research into therapeutic processes
The cost-effectiveness of the approach tool for therapy training, research, and service development, offering risk-free experimentation and learning opportunities, became transparent during its implementation. Human-to-human alternatives would take years and millions of pounds to gather data and years to analyse. The above output was achieved in 7 days.
The approach's future strengths may lay in its capability to simulate other environments
1. Complex therapeutic interventions
2. Explore challenging clinical scenarios e.g. Human-to-AI Agent
3. Test different therapeutic approaches
4. Develop and refine treatment protocols
5. Train new therapists in a safe environment
Future Work
Therapists involved in the work concluded that there is some value in forming a community of practice to explore approaches like DioSim. To this end, CarefulAI has committed to support its development.
The steps associated with the practice development are understanding the value of using multi-agent modelling systems like TinyTroupe to explore decreasing costs in the NHS.
Specifically, the time and expense of people outside CarefulAI setting up such simulations and doing the same analysis.
Given the complexity of the NHS and its devolved nature across Integrated Care Systems (ICSs), the community of practice will also look at ways to use DioSim and multi-agent systems to:
- Identify ways to reduce costs in clinical processes
- Develop ways in which it can be used to focus the efforts of those involved in digital transformation
- Develop best practice guidelines around the use of AI Agents in system modelling
- Form integration strategies for those wishing to plan for the impact of co-pilots and agents.
- Create governance structures for the use of agents involved in managing human-AI agent interfaces
- Develop risk management frameworks for the use of dialogue agents, avatars and digital twins
This approach aligns with NHS Long Term Plan objectives and supports the broader digital transformation agenda whilst maintaining focus on improved patient outcomes and service quality.
It is envisaged that the community of practice in mental health would look as follows:
- Identify ways to reduce costs in clinical processes
- Develop ways in which it can be used to focus the efforts of those involved in digital transformation
- Develop best practice guidelines around the use of AI Agents in system modelling
- Form integration strategies for those wishing to plan for the impact of co-pilots and agents.
- Create governance structures for the use of agents involved in managing human-AI agent interfaces
- Develop risk management frameworks for the use of dialogue agents, avatars and digital twins
This approach aligns with NHS Long Term Plan objectives and supports the broader digital transformation agenda whilst maintaining focus on improved patient outcomes and service quality.
It is envisaged that the community of practice in mental health would look as follows:
