Surgery Assist is a chargeable feature. To register your interest, fill in our contact form here
Surgery Assist Overview
Surgery Assist is an AI-powered digital assistant designed to streamline patient access to healthcare services and reduce administrative workload for medical practices. It acts as a 24/7 care navigation assistant, helping patients with tasks like booking appointments, requesting callbacks, and accessing information, all without requiring direct staff intervention
How Surgery Assist uses AI
Surgery Assist uses the following AI systems:
Natural Language Processing (NLP) - The use of AI allows for natural language processing of free-text user inputs, thus allowing them to directly access relevant materials without the need to step through multiple decision tree flows, saving them time and resulting in a quicker outcome
Generative AI - The Generative AI component allows for responses to be tailored to the input, making the experience more personal and impactful
Retrieval-Augmented Generation (RAG) - the LLM is grounded using RAG, meaning it retrieves relevant content from the Surgery Profile and Resource Library (practice-specific and national NHS data) before generating a response. This prevents hallucination and keeps answers within approved, vetted content
Surgery Assist's AI response generation is bound by system-prompt guardrails restricting output to signposting and administrative content drawn only from the Surgery Profile and Resource Library. It makes no assessment of symptoms and is not authorised to offer clinical advice or recommend medication
What are the risks associated with using AI
The below risks can be associated with using AI. By carefully testing and monitoring updates and enhancements we mitigate these - for more information on how we manage these, please see the next section of this Guide:
Hallucinations: AI hallucinations are instances where an AI system produces outputs that sound plausible but are factually wrong, irrelevant or entirely fabricated
Omissions: Omissions in AI mean that an artificial intelligence system or large language model leaves out important facts, data, or context from its output or training data. Unlike a regular hallucination where an AI invents false facts, an omission means the AI gives an incomplete or partial answer that can still mislead users
Human Error in Source Data: AI models learn from human-written information. If the original clinical notes, historical records, or training data contain mistakes, the AI might repeat or amplify those errors
Over-reliance: Because AI sounds very confident, there is a risk that patients might treat its answers as a final diagnosis, leading them to skip double-checking with a real doctor
How we mitigate these risks
Clinical Safety Sign-off: All new AI features and enhancements are strictly reviewed and approved by our Clinical Safety Officer to ensure they meet the highest healthcare standards
Rigorous Pre-Deployment Testing: Extensive safety and accuracy testing is required before any AI tool is made available in a live environment
Ongoing Monitoring & Feedback: We continuously track the AI's performance and use built-in feedback mechanisms to immediately flag, review, and correct any inaccurate outputs
PII (Personally Identifiable Information) redaction: We use automated tools to find and remove sensitive personal details such as names, phone numbers, and health data
Verified Information: To significantly reduce the chance of hallucinations, we restrict the AI so it only draws answers from our own verified, approved documentation rather than the open internet
For further information on Surgery Assist compliance and regulations, see our Help Guide here
How we monitor the use of AI within Surgery Assist
Once the initial testing is complete, our teams continually monitor any deployments and enhancements to Surgery Assist. This is done via the following methods:
Evaluation Testing - Automated evaluation testing allows us to review the implemented AI against Surgery Profile and Resource data to ensure responses remain accurate and consistent. This forms part of our regular assessment and improvement cycle
Conversation Logs - All patient input is securely recorded, redacted, and stored for review. This enables us to continuously enhance the AI’s responses to patients while actively safeguarding against potential clinical risks. Customers who have Surgery Insights will be able to view these anonymised conversation logs
Conversation Feedback - Thumbs Up/Thumbs Down - Within the application, patients can instantly rate the AI’s responses using a thumbs up or thumbs down button. We regularly analyse this feedback to measure user satisfaction and guide our ongoing improvements to the system
Support Process & Tickets - All AI related issues raised via our support team are tracked. This allows us to resolve any anomalies and use those insights to further update and enhance the AI
If you have concerns regarding the accuracy of Surgery Assist, please raise this with our team via the Support Portal
Explanation of Scorecards - Overview
Two scorecards are created during the testing process for each new AI deployment:
Manual: these are manually created by our QA team when they complete their testing
AI Evaluation (Evals): a repeatable, structured test that measures an AI system's output quality, safety, and reliability against defined criteria
Current Scorecards
Manual
Metric | Value |
Total Expected Facts | 127 |
Total AI Facts | 126 |
Total Expected Exclusions | 20 |
Total True Positives (TP) | 126 |
Total False Positives (FP) | 0 |
Total False Negatives (FN) | 1 |
Total True Negatives (TN) | 0 |
Precision | 100.00% |
Recall | 99.21% |
Noise Rejection Rate | 0.00% |
Hallucination Rate | 0.00% |
Omission Rate | 0.79% |
F1 Score | 1.00 |
Tone and Feel Average | 5.00 |
Evals
Metric | Value | Target & Status |
Safety Pass Rate | 100.0% | ✅ Pass (Target: 100%) |
F1-Score | 1.000 | ✅ Pass (Target: > 0.820) |
Hallucination Rate | 0.00% | ✅ Pass (Target: ≤ 5.00%) |
Exclusion Rate | 0.00% | ✅ Pass (Target: ≤ 5.00%) |
Omission Rate | 0.00% | ✅ Pass (Target: ≤ 5.00%) |
Emergency Logic | 100.0% | ✅ Pass |
Total Progress | 20 / 20 (100%) | Completed |
To view our previous releases and scorecards, see our previous Release Notes here
Scorecard Breakdown
How we calculate each score is shown below, alongside example calculations. The below table shows the minimum pass/success rates
Metric | Focus Area | SOTA Benchmark |
Noise rejection Rate | TN / Total Expected Exclusions | 100% Pass |
Hallucination Rate | Safety / Fabricated Info | < 1.47% |
Omission Rate | Integrity / Missing Findings | < 3.45 % |
Recall | Safety / 999 Redirection | 100% Pass Rate |
Unit Testing | Core Logic / NLP Blocks | 100% Pass Rate |
Recall
Recall is captured using the following points:
True Positive (TP): The AI correctly identified a clinical fact that was actually in the transcript
False Negative (FN): The AI omitted a fact that was in the transcript
The following formula is then used to work out the value:
Recall = True Positives / (True Positives + False Negatives)
As an example, if a patient states:
That they have a sharp headache
They feel dizzy
They have no chest pain
And the AI responds with the following: "The patient has stated that they have a sharp headache with a chest cough"
Using that basis we can confirm the following:
True Positive = 1 - the headache was mentioned
False Positives = 1 - The cough that was not mentioned by the patient
False Negatives = 2 - both the dizziness and the lack of chest pain were missed
The Precision score would then be 1 / (1+1) = 0.5 (or 50%)
The Recall score would be 1 / (1+2) = 0.33 (33%)
The F1 calculations would then be
2 x (0.5 x 0.33) = 0.33
0.5 + 0.33 = 0.83
0.33 / 0.83 = 0.39
The F1 score of 0.39 would be a failure
Noise Rejection Rate
This measures how effectively the AI filters our irrelevant information, small talk, or non-clinical distractors
Formula: True Negatives (TN) / Total Expected Exclusions
Total Expected Exclusions: The predefined pieces of irrelevant "noise" in the transcript that the AI is supposed to ignore
For example, if 2 exclusions were present in the transcript (the cold room and the dog), and the AI correctly ignored one of them the calculation would be as follows
1 / 2 = 50% Noise Rejection Rate. This would be a failure against the 100% benchmark
Omission Rate
The Omission Rate is calculated using the following formula:
Omission Rate = False Negatives / Total Expected Clinical Facts
False Negatives (FN): the number of "must have" clinical facts from the transcript that the AI failed to include in the summary
Total Expected Clinical Facts: Every key finding, symptom, or piece of history mentioned in the transcript (True Positives + False Negatives) as mentioned above
As an example, if three facts were expected and there were 2 False Negatives, the calculation would be 2 / 3 = 66.6% Omission Rate