UXAI
Alignment of the User Experience of an AI tool with human good practice is important. We call our approach UXAI. It is best explained by example:
UXAI and Large Language Models (LLMs)
LLMs are made available across the Internet and in some cases are embodied in other tools e.g. ChatGPT, Microsoft Office et
Prior to LLMs being used it was hard to draft text content. Via them people can now more easily answer questions, draft emails, or other pieces of writing, write code, answer questions about a set of documents, create conversational agents, give software a natural language interface, tutor in a range of subjects, translate languages and even simulate characters for video games.
Prior to using a particular LLM it is wise that end users validate their fitness for purpose to themselves, their professions and their employers. This can be done in many ways, via workshops, events or online.
UXAI Alignment involves groups of six or more people developing a plan to optimise benefits and mitigate risks associated with a LLM.
After describing
People are then encourage to test the LLM. This is not done with PPI data. It is also done in a manner that protects the testers privacy and the organisations for whom they work. Tests are conducted
People are then encouraged to develop questions they need to raise with their professions and employers. These are them discussed and groups are asked to rank the importance of themes arising in the questions created.
To assist the creation of a risk register of themes arising from questions PRIDAR is used. An example of a risk register is shown below.
Prior to LLMs being used it was hard to draft text content. Via them people can now more easily answer questions, draft emails, or other pieces of writing, write code, answer questions about a set of documents, create conversational agents, give software a natural language interface, tutor in a range of subjects, translate languages and even simulate characters for video games.
Prior to using a particular LLM it is wise that end users validate their fitness for purpose to themselves, their professions and their employers. This can be done in many ways, via workshops, events or online.
UXAI Alignment involves groups of six or more people developing a plan to optimise benefits and mitigate risks associated with a LLM.
After describing
- how to use the LLM
- issues around effective use of the LLM
- how the LLM retains data to learn from the user
People are then encourage to test the LLM. This is not done with PPI data. It is also done in a manner that protects the testers privacy and the organisations for whom they work. Tests are conducted
- Without prompts
- With Prompts
- With Prompt and changes in other parameters e.g. temperatures
- With and without third party interfaces
People are then encouraged to develop questions they need to raise with their professions and employers. These are them discussed and groups are asked to rank the importance of themes arising in the questions created.
To assist the creation of a risk register of themes arising from questions PRIDAR is used. An example of a risk register is shown below.
Testers then develop a plan. The plan relates to giving the LLM provider, their profession and their employer feedback on the UXAI events outputs, and raising the questions arising with those who may be able to address them. They are then actively encouraged to run their own UXAI events, and encourage others to do the same.
For LLMs, in our experience, it is the questions arising that prove to be the most useful for all concerned
For LLMs, in our experience, it is the questions arising that prove to be the most useful for all concerned
CarefulAI PRIDAR users can save the output of their staffs deliberations around LLMs using the form below
UXAI and Suicide Prevention AI
SISDA enables clinicians to be alerted to suicidal ideation language in text (see Model Feedback). Without SISDA clinicians would need to look at thousands of self referral requests each week.
Prior to SISDA being used, it is validated by mental health clinicians as fit for purpose. This ensures its use is aligned with good practice. For background on SISDA please see its model label under 'Motives and Models'
In SISDAs case the users are clinicians. The intent being considered is suicidal ideation in text.
UXAI Alignment involves groups of three or more senior clinicians validating the fitness for purpose of the SISDA's outputs.
They do this by annotating the output of an AI approach with one of the following labels:
For example:
Prior to SISDA being used, it is validated by mental health clinicians as fit for purpose. This ensures its use is aligned with good practice. For background on SISDA please see its model label under 'Motives and Models'
In SISDAs case the users are clinicians. The intent being considered is suicidal ideation in text.
UXAI Alignment involves groups of three or more senior clinicians validating the fitness for purpose of the SISDA's outputs.
They do this by annotating the output of an AI approach with one of the following labels:
- To be Validated
- Evidence of possible Intent
- Campaigns relating to the Intent
- Flippant Reference to the Intent
- Information or Support for the Intent
- Memorial or Condolence for the Intent
- Reporting of the Intent
- None of the above
For example:
