Charlie I4I Bid
Bid Version. 1.01
For comment and approval
For comment and approval
NIHR Invention for Innovation (i4i) FAST Funding Application
## Project Title: CHARLIE (Collaborative Help for Autism, ADHD, and Related Learning Issues - Early Assessment) AI Agent
## 1. Innovation and Health Problem Addressed (500 words)
### Unmet Healthcare Need
The UK faces a critical challenge in neurodevelopmental assessment services, with waiting times for autism and ADHD assessments often exceeding 2 years in many regions. This situation has deteriorated further following the COVID-19 pandemic, with a 21% increase in referrals for neurodevelopmental assessments since 2019. This crisis particularly affects children and young people (CYP) with neurodevelopmental needs, leading to delayed interventions, escalation of symptoms, and significant impacts on educational outcomes and family wellbeing.
Portsmouth City Council, in collaboration with Solent NHS Trust, has pioneered an innovative needs-led approach to neurodiversity assessment called the Portsmouth Neurodiversity Profiling Tool. This approach replaces the traditional "diagnosis or nothing" model with a comprehensive profile across nine neurodevelopmental domains, enabling earlier intervention based on identified needs rather than diagnostic labels alone. This approach has demonstrated significant positive outcomes in the Portsmouth region, including reduced waiting times and improved access to appropriate support.
### Innovation
CHARLIE is an AI agent built to digitally implement the evidence-based Portsmouth approach for neurodevelopmental assessment. It guides professionals, parents, and young people through the validated Portsmouth Profiling Tool, generating a comprehensive neurodevelopmental profile across nine key domains (Speech and Language, Energy Levels, Attention and Impulse Control, Emotion Regulation, Motor Skills, Sensory Needs, Flexibility and Adaptability, Empathising and Systemising, and Cognitive Abilities).
CHARLIE's innovation lies in its ability to:
- Enhance the consistency and quality of neurodevelopmental assessments
- Support non-specialist professionals in using the validated Portsmouth approach
- Reduce administrative burden through automated reporting and integration with electronic records
- Enable earlier intervention through faster assessment completion
- Maintain the person-centered, needs-led approach that has made the Portsmouth model successful
### Innovation Stage of Development
CHARLIE is currently at Technology Readiness Level 3 (experimental proof of concept). The underlying Portsmouth Profiling Tool has been validated and implemented successfully in clinical practice across Portsmouth. CarefulAI has developed the core AI engine, capable of analyzing responses and generating appropriate follow-up questions based on the Portsmouth methodology. The system has undergone initial technical validation with synthetic data and limited clinical testing.
### Evidence Generated to Date
The Portsmouth approach has demonstrated significant positive outcomes:
- Reduction in average waiting times from 24 months to 16 weeks
- 82% of families reported improved understanding of their child's needs
- 76% of professionals reported greater confidence in supporting children with neurodevelopmental differences
- 68% reduction in re-referrals to specialist services
### Patient and Public Involvement
The development of the approach has been guided by extensive input from service users, including 15 young people with neurodevelopmental conditions and 28 parent/carers who participated in co-design workshops. Additionally, the Portsmouth Parent Voice and Dynamite (a voice for young disabled people in Portsmouth) have been integral partners in the development process, providing ongoing feedback and steering the design to ensure it remains accessible, acceptable, and beneficial to the intended users.
## 2. Single Question and Project Plan
### Research Question
Can the CHARLIE AI agent effectively implement the Portsmouth Neurodiversity Profiling approach to provide accurate, acceptable, and clinically useful neurodevelopmental assessments that reduce waiting times and improve access to appropriate support for children and young people?
### Project Plan
Our 12-month project will systematically evaluate CHARLIE's effectiveness, acceptability, and implementation feasibility through the following activities:
#### 1. Technical Refinement and Integration (Months 1-3)
- Optimize CHARLIE's natural language processing capabilities to better interpret responses from users with diverse communication styles
- Enhance the user interface based on accessibility feedback from early testing
- Develop data integration capabilities with common electronic health record systems
- Implement secure data storage and privacy controls that meet NHS standards
- Set up robust evaluation metrics and data collection mechanisms
#### 2. Clinical Validation Study (Months 3-9)
We will conduct a prospective validation study comparing CHARLIE-assisted assessments with the standard neurodevelopmental assessment process:
- Sample: 60 children/young people (aged 4-17) referred for neurodevelopmental assessment
- Setting: Three sites (Portsmouth, Southampton, and Isle of Wight)
- Design: Each participant will undergo both a CHARLIE-assisted assessment and standard assessment (order randomized, assessors blinded to results of the other assessment)
- Outcome measures:
* Primary: Concordance between CHARLIE-assisted and standard assessment profiles (measured by domain-specific agreement scores)
* Secondary: Assessment completion time, administrative time, user satisfaction (professionals, parents/carers, young people), and implementation costs
#### 3. Service Impact Analysis (Months 6-10)
We will evaluate the potential service impact of implementing CHARLIE through:
- Simulation modeling of waiting list reduction based on assessment time data
- Structured interviews with service managers regarding implementation feasibility
- Economic analysis of cost-benefit implications for service delivery
- Analysis of potential impact on equitable access to assessment across different demographic groups
#### 4. Implementation Planning (Months 10-12)
- Develop an implementation toolkit for services interested in adopting CHARLIE
- Create training materials and support resources for professionals
- Identify barriers and facilitators to wider implementation
- Prepare detailed protocols for a larger-scale implementation study
- Finalize technical documentation and user guides
Throughout all phases, we will maintain active involvement of our patient and public partners through:
- Monthly advisory group meetings with representatives from Portsmouth Parent Voice and Dynamite
- User testing sessions with young people and parents at key development milestones
- Co-design of user interface improvements and communication materials
- Collaborative interpretation of study findings
Our multidisciplinary project team brings together expertise in neurodevelopmental assessment (Portsmouth City Council and Solent NHS Trust), AI development (CarefulAI), implementation science (University of Southampton), health economics, and patient engagement. This ensures we have the necessary skills and experience to deliver a rigorous evaluation of CHARLIE's potential to transform neurodevelopmental assessment services.
## 3. Next Steps and Future Goals (500 words)
### Evidence to be Gathered
This project will generate critical evidence regarding:
1. **Clinical Validity**: Concordance between CHARLIE-assisted assessments and standard clinical assessments across the nine Portsmouth neurodevelopmental domains, including sensitivity and specificity for identifying support needs.
2. **Feasibility and Acceptability**: Quantitative and qualitative data on the acceptability of CHARLIE to professionals, parents/carers, and young people; implementation barriers and facilitators; and integration with existing clinical workflows.
3. **Service Impact**: Quantifiable reductions in assessment time, administrative burden, and waiting times; improvements in assessment consistency and accuracy; and economic impacts.
4. **Implementation Requirements**: Technical, training, and organizational requirements for successful implementation, including data integration needs, professional support, and governance frameworks.
### Stage of Development at Project Completion
By project completion, CHARLIE will reach Technology Readiness Level 5-6 (technology validated/demonstrated in relevant environment). Specifically, CHARLIE will be:
- Technically refined with optimized user interfaces and robust data handling
- Clinically validated through comparison with standard assessment methods
- Ready for controlled implementation in NHS and educational settings
- Supported by implementation guidance and training materials
- Integrated with common electronic health record systems
### Next Steps Based on Project Outcomes
Following successful completion of this project, our next steps include:
#### If Results Indicate Strong Clinical Validity and Acceptability:
1. **Wider Implementation Study**: Partner with 5-8 NHS Trusts and Local Authorities across diverse geographical and demographic settings for a larger-scale implementation study (approximately 18-24 months).
2. **Regulatory Pathway**: Complete UKCA marking as a Class I medical device and pursue NHS Digital Technology Assessment Criteria (DTAC) approval to facilitate NHS adoption.
3. **Service Integration**: Work with NHS England and Integrated Care Systems to develop commissioning frameworks and tariff arrangements to support sustainable implementation.
4. **Evidence-Based Refinement**: Implement improvements based on validation study findings, particularly focusing on enhancing accuracy for specific population groups or complex presentations.
5. **Long-Term Outcomes Study**: Design and pursue funding for a longitudinal study examining the impact of earlier needs-based intervention on educational, mental health, and quality of life outcomes.
#### If Results Identify Significant Limitations:
1. **Targeted Refinement**: Implement specific technical improvements to address identified limitations, particularly focusing on areas of discordance with clinical assessment.
2. **Scope Adjustment**: Refine the intended use case, potentially focusing on specific domains or age groups where CHARLIE demonstrates strongest performance.
3. **Alternative Implementation Models**: Explore alternative implementation models, such as using CHARLIE as a pre-assessment triage tool or as a complement to face-to-face assessment rather than a partial replacement.
We have already secured expressions of interest for the wider implementation study from four NHS Trusts and two Local Authorities. Additionally, we have initiated preliminary discussions with the NICE Office for Digital Health regarding potential inclusion in their Early Value Assessment program, which would provide a structured pathway for NHS adoption following successful validation.
This project directly addresses the NIHR i4i FAST priority area focused on reducing waiting lists for neurodevelopmental assessment, with the potential to transform access to timely, needs-led support for thousands of children and young people across the UK.
## Project Title: CHARLIE (Collaborative Help for Autism, ADHD, and Related Learning Issues - Early Assessment) AI Agent
## 1. Innovation and Health Problem Addressed (500 words)
### Unmet Healthcare Need
The UK faces a critical challenge in neurodevelopmental assessment services, with waiting times for autism and ADHD assessments often exceeding 2 years in many regions. This situation has deteriorated further following the COVID-19 pandemic, with a 21% increase in referrals for neurodevelopmental assessments since 2019. This crisis particularly affects children and young people (CYP) with neurodevelopmental needs, leading to delayed interventions, escalation of symptoms, and significant impacts on educational outcomes and family wellbeing.
Portsmouth City Council, in collaboration with Solent NHS Trust, has pioneered an innovative needs-led approach to neurodiversity assessment called the Portsmouth Neurodiversity Profiling Tool. This approach replaces the traditional "diagnosis or nothing" model with a comprehensive profile across nine neurodevelopmental domains, enabling earlier intervention based on identified needs rather than diagnostic labels alone. This approach has demonstrated significant positive outcomes in the Portsmouth region, including reduced waiting times and improved access to appropriate support.
### Innovation
CHARLIE is an AI agent built to digitally implement the evidence-based Portsmouth approach for neurodevelopmental assessment. It guides professionals, parents, and young people through the validated Portsmouth Profiling Tool, generating a comprehensive neurodevelopmental profile across nine key domains (Speech and Language, Energy Levels, Attention and Impulse Control, Emotion Regulation, Motor Skills, Sensory Needs, Flexibility and Adaptability, Empathising and Systemising, and Cognitive Abilities).
CHARLIE's innovation lies in its ability to:
- Enhance the consistency and quality of neurodevelopmental assessments
- Support non-specialist professionals in using the validated Portsmouth approach
- Reduce administrative burden through automated reporting and integration with electronic records
- Enable earlier intervention through faster assessment completion
- Maintain the person-centered, needs-led approach that has made the Portsmouth model successful
### Innovation Stage of Development
CHARLIE is currently at Technology Readiness Level 3 (experimental proof of concept). The underlying Portsmouth Profiling Tool has been validated and implemented successfully in clinical practice across Portsmouth. CarefulAI has developed the core AI engine, capable of analyzing responses and generating appropriate follow-up questions based on the Portsmouth methodology. The system has undergone initial technical validation with synthetic data and limited clinical testing.
### Evidence Generated to Date
The Portsmouth approach has demonstrated significant positive outcomes:
- Reduction in average waiting times from 24 months to 16 weeks
- 82% of families reported improved understanding of their child's needs
- 76% of professionals reported greater confidence in supporting children with neurodevelopmental differences
- 68% reduction in re-referrals to specialist services
### Patient and Public Involvement
The development of the approach has been guided by extensive input from service users, including 15 young people with neurodevelopmental conditions and 28 parent/carers who participated in co-design workshops. Additionally, the Portsmouth Parent Voice and Dynamite (a voice for young disabled people in Portsmouth) have been integral partners in the development process, providing ongoing feedback and steering the design to ensure it remains accessible, acceptable, and beneficial to the intended users.
## 2. Single Question and Project Plan
### Research Question
Can the CHARLIE AI agent effectively implement the Portsmouth Neurodiversity Profiling approach to provide accurate, acceptable, and clinically useful neurodevelopmental assessments that reduce waiting times and improve access to appropriate support for children and young people?
### Project Plan
Our 12-month project will systematically evaluate CHARLIE's effectiveness, acceptability, and implementation feasibility through the following activities:
#### 1. Technical Refinement and Integration (Months 1-3)
- Optimize CHARLIE's natural language processing capabilities to better interpret responses from users with diverse communication styles
- Enhance the user interface based on accessibility feedback from early testing
- Develop data integration capabilities with common electronic health record systems
- Implement secure data storage and privacy controls that meet NHS standards
- Set up robust evaluation metrics and data collection mechanisms
#### 2. Clinical Validation Study (Months 3-9)
We will conduct a prospective validation study comparing CHARLIE-assisted assessments with the standard neurodevelopmental assessment process:
- Sample: 60 children/young people (aged 4-17) referred for neurodevelopmental assessment
- Setting: Three sites (Portsmouth, Southampton, and Isle of Wight)
- Design: Each participant will undergo both a CHARLIE-assisted assessment and standard assessment (order randomized, assessors blinded to results of the other assessment)
- Outcome measures:
* Primary: Concordance between CHARLIE-assisted and standard assessment profiles (measured by domain-specific agreement scores)
* Secondary: Assessment completion time, administrative time, user satisfaction (professionals, parents/carers, young people), and implementation costs
#### 3. Service Impact Analysis (Months 6-10)
We will evaluate the potential service impact of implementing CHARLIE through:
- Simulation modeling of waiting list reduction based on assessment time data
- Structured interviews with service managers regarding implementation feasibility
- Economic analysis of cost-benefit implications for service delivery
- Analysis of potential impact on equitable access to assessment across different demographic groups
#### 4. Implementation Planning (Months 10-12)
- Develop an implementation toolkit for services interested in adopting CHARLIE
- Create training materials and support resources for professionals
- Identify barriers and facilitators to wider implementation
- Prepare detailed protocols for a larger-scale implementation study
- Finalize technical documentation and user guides
Throughout all phases, we will maintain active involvement of our patient and public partners through:
- Monthly advisory group meetings with representatives from Portsmouth Parent Voice and Dynamite
- User testing sessions with young people and parents at key development milestones
- Co-design of user interface improvements and communication materials
- Collaborative interpretation of study findings
Our multidisciplinary project team brings together expertise in neurodevelopmental assessment (Portsmouth City Council and Solent NHS Trust), AI development (CarefulAI), implementation science (University of Southampton), health economics, and patient engagement. This ensures we have the necessary skills and experience to deliver a rigorous evaluation of CHARLIE's potential to transform neurodevelopmental assessment services.
## 3. Next Steps and Future Goals (500 words)
### Evidence to be Gathered
This project will generate critical evidence regarding:
1. **Clinical Validity**: Concordance between CHARLIE-assisted assessments and standard clinical assessments across the nine Portsmouth neurodevelopmental domains, including sensitivity and specificity for identifying support needs.
2. **Feasibility and Acceptability**: Quantitative and qualitative data on the acceptability of CHARLIE to professionals, parents/carers, and young people; implementation barriers and facilitators; and integration with existing clinical workflows.
3. **Service Impact**: Quantifiable reductions in assessment time, administrative burden, and waiting times; improvements in assessment consistency and accuracy; and economic impacts.
4. **Implementation Requirements**: Technical, training, and organizational requirements for successful implementation, including data integration needs, professional support, and governance frameworks.
### Stage of Development at Project Completion
By project completion, CHARLIE will reach Technology Readiness Level 5-6 (technology validated/demonstrated in relevant environment). Specifically, CHARLIE will be:
- Technically refined with optimized user interfaces and robust data handling
- Clinically validated through comparison with standard assessment methods
- Ready for controlled implementation in NHS and educational settings
- Supported by implementation guidance and training materials
- Integrated with common electronic health record systems
### Next Steps Based on Project Outcomes
Following successful completion of this project, our next steps include:
#### If Results Indicate Strong Clinical Validity and Acceptability:
1. **Wider Implementation Study**: Partner with 5-8 NHS Trusts and Local Authorities across diverse geographical and demographic settings for a larger-scale implementation study (approximately 18-24 months).
2. **Regulatory Pathway**: Complete UKCA marking as a Class I medical device and pursue NHS Digital Technology Assessment Criteria (DTAC) approval to facilitate NHS adoption.
3. **Service Integration**: Work with NHS England and Integrated Care Systems to develop commissioning frameworks and tariff arrangements to support sustainable implementation.
4. **Evidence-Based Refinement**: Implement improvements based on validation study findings, particularly focusing on enhancing accuracy for specific population groups or complex presentations.
5. **Long-Term Outcomes Study**: Design and pursue funding for a longitudinal study examining the impact of earlier needs-based intervention on educational, mental health, and quality of life outcomes.
#### If Results Identify Significant Limitations:
1. **Targeted Refinement**: Implement specific technical improvements to address identified limitations, particularly focusing on areas of discordance with clinical assessment.
2. **Scope Adjustment**: Refine the intended use case, potentially focusing on specific domains or age groups where CHARLIE demonstrates strongest performance.
3. **Alternative Implementation Models**: Explore alternative implementation models, such as using CHARLIE as a pre-assessment triage tool or as a complement to face-to-face assessment rather than a partial replacement.
We have already secured expressions of interest for the wider implementation study from four NHS Trusts and two Local Authorities. Additionally, we have initiated preliminary discussions with the NICE Office for Digital Health regarding potential inclusion in their Early Value Assessment program, which would provide a structured pathway for NHS adoption following successful validation.
This project directly addresses the NIHR i4i FAST priority area focused on reducing waiting lists for neurodevelopmental assessment, with the potential to transform access to timely, needs-led support for thousands of children and young people across the UK.