An Analysis of the LUNA Data Quality Confidence Score

You may be familiar with the LUNA data quality confidence score applied to your Trust, but is it an accurate representation of your Trust’s PTL accuracy?

There are 12 indicators that comprise the LUNA data quality confidence score. We analysed the logic behind the indicators as well as applying them to 1 million pathways to see the results.

Six of the indicators relate to pathways that you have already identified are not subject to RTT reporting. These are pathways that would never be included in your monthly RTT returns. These indicators are:

1. Non-RTT code 21

2. Non-RTT codes 30-36

3. Non-RTT codes 90, 91

4. Non-RTT code 98

5. Non-RTT code 99

6. Non-RTT Treatment Function or missing Treatment function

This means pathways are flagged as errors, despite being correctly recorded as Non-RTT by the Trust. You would certainly exclude these from your monthly RTT submissions as a matter of course.

The remaining 6 indicators are one-dimensional and can generate significant false positive results. These are:

7. Potential duplicates different clock start date

8. Potential duplicates same clock start date

   
   

Approximately 73% of all duplicates are false positives (they are genuine simultaneous pathways – very common in both T&O and Ophthalmology).

9. Outcome of discharged - approximately 52% false positives (frequently due to discharge from clinic or a particular consultant but ongoing care within the Trust is still required)

10. Source of Referral ED - approximately 65% false positives (the patient is correctly on an active RTT pathway)

11. Admission method planned or invalid (Code 13) – the majority are false positives, as code 13 is commonly applied whenever a TCI date is set even if it is an RTT pathway

12. Over 40 weeks and not validated – the majority are false positives, meaning these are mostly correctly active RTT pathways

Based upon the pathways that trigger these indicators, you are provided with an overall DQ confidence score (e.g. 98%).

Here are questions you may want to ask about this score.

• We estimate that between 30%-50% of pathways on a typical PTL will trigger one of the indicators listed above

  • Are all the pathways that trigger an indicator considered to be on the PTL in error?

  • Conversely, are all the pathways that do not trigger an indicator considered to be correctly on the RTT PTL?

• When my Trust validates the pathways flagged in LUNA as a DQ error, what percentage of these do we actually remove?

• How exactly is the Data Quality Confidence score calculated and weighted?

• Looking at your own data, does your LUNA DQ confidence score correspond with the volume of pathways you removed during the sprints? Or the known data quality issues you have?

• How does your DQ confidence score compare to that of other Trusts? What does good look like?

• Has this DQ score helped you better understand your PTL accuracy and how to improve it?

Given that the LUNA data quality confidence score may be used as a metric for your PTL accuracy, we believe it is important that the methodology is transparent and open to scrutiny.

By comparison, here is how Source Group evaluates your PTL accuracy.

We use our industry leading artificial intelligence (AI) solution to conduct a Gold-standard, in depth statistical analysis of every pathway.

More than 40 different attributes of each pathway are examined simultaneously, not just a single indicator. For example, the AI may see that for an individual pathway:

• Weeks wait is above the treatment functions polling range

• Patient has had an attendance

• 75% of patients within the treatment function are discharged at 1st appointment

• Patient does not have a further TCI date

• It’s been 6 weeks since the last attendance with no further activity

• Pathway has not yet been validated

• RTT status code has been changed from a 10 to a 20

  
  

It will then test the presence of these factors against a pathway outcomes data set of >4 million pathways as well as factoring in your Trust’s nuances and each Treatment Function’s nuances.

From this, each pathway will be given a percentage chance it should not be subject to RTT reporting, e.g., the pathway has a 90% chance of removal.

The sum of these individual probabilities across the entire PTL is the Trust’s inflation score.

An example output of this analysis is below:

This has proven to be an incredibly effective way of determining the accuracy of your RTT PTL, as well as to target validations for removal.

Our inflation model has been shown to be 99.5% accurate during last year’s validation sprints.

This analysis will support your Trust:

• Improve your RTT performance

• Understand the level of inflation present on your PTL

• Understand which Treatment Functions have the highest levels of inflation

• Provide the foundation for a strategy to effectively and sustainably reduce inflation

  
  

We are offering all Trusts a no-cost analysis of their RTT PTL. All we need to do so is to be sent your Trust’s most recent WLMDS submission – no systems integration or software installation is required.

Upon receiving your WLMDS, we can provide you with an analysis the very next day.

If you would like to take us up on our offer of a no-cost analysis of your RTT PTL, please contact us