Statistical Process Control Methodology and Statistical Significance
Spine charts for the indicators have been created using statistical process control (SPC) principles and use control limits to indicate variation from the national mean. Values that lie within these limits are said to display ‘normal cause variation’ in that the variation from the national mean can be considered to be random. Values that lie outside of these limits are said to display ‘special cause variation’ and a cause other than random chance should be considered.
A value that lies outside of the two standard deviation (SD) control limits can be considered as an ‘alert’. Values that lie outside 3SD can be viewed as ‘alarms’.
Calculation of control limits
Indicators on the dashboard have control limits of 95% (2 standard deviations) and 99.8% (3 standard deviations). These are calculated using the Wilson Score Method for proportions, rates and SMR’s 1.
An excel tool for this method can be found at http://www.apho.org.uk/resource/item.aspx?RID=48457.
Definitions for standardisation have been taken from Association of Public Health Observatories (APHO) Technical Briefing 3 (March 2008) ‘Commonly used public health statistics and their confidence intervals’2.
Where indicators have been standardised for age and sex age quintiles up to 85+ were used.
Indirect Standardised Rate
A set of reference age-specific rates are used for indirect age standardisation. In nearly all instances these are real rates rather than hypothetical ones and are taken from the reference or benchmark data against which the populations being standardised are to be compared. These are often expressed as rate of observed over expected events.
Suppression of small numbers
Methods Analytics Ltd follows the Hospital Episode Statistics (HES) Analysis Guide and suppresses any numerators, denominators, and values if appropriate, where the value is less than six to ensure any data is not identifiable. We apply these suppression rule at source before any data is published to Stethoscope.
In addition to this, where we have stated numerator and denominator are both <=5 and the value is a percentage such as 25% we will suppress the value to prevent back calculation of the suppressed numbers.
Where we present counts the value and numerator are the same and there is no denominator so we suppress if count <=5.
Metrics in Stethoscope that are using Directly Standardised Rates (DSR) are not suppressed as we only hold the DSR per 100,000 population and the population value itself.
Spine charts are a way of displaying variation data that is derived from a funnel plot. A funnel plot shows data for a range of organisations at a single point in time. The denominator (count of activity, population etc) is plotted on the X axis and the value of the measure (mortality rate, readmission rate) on the Y axis.
For a single organisation we know what the X axis (denominator) value is, as this is part of the data set. Therefore we can take a vertical slice through the funnel plot:
Rotating this creates the spine chart. The elements of which are:
Where the trust value is within the grey central portion of the chart the performance on this indicator does not differ from the national mean by more than can be explained by random chance.
Some metrics may have grey/blue ranges replacing the green and red. This is where the indicator does not have a perceived desired direction for improvement. Where this occurs trusts may still fall within the 2/3SD limits which suggests their rate is normal, or outside of the 2/3SD limits which shows the rate for the trust cannot be explained by random chance.
Variable Life Adjusted Display (VLAD)A VLAD plot provides a means to understand whether the outcome is operating within expected ‘normal’ variation. They allow you to answer the question ‘Do I have a problem with mortality in my Hospital?’ and provide an alert, or signal when mortality is rising (or falling) faster than statistically expected.
The VLAD has achieved some popularity among clinicians by producing clear displays of outcome histories that they can easily relate to. Mathematically, it is a plot of the cumulative difference between the expected and actual outcomes over as series of patients. So if Xn denotes the outcome of the nth patient and Vn the corresponding risk.
the VLAD plot position is calculated by the formula:
 Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc1927;22:209-12