National hospital performance surveillance system

Our work on appropriate statistical techniques in mortality surveillance in general practice, established the utility of statistical process control charts in surveillance of healthcare performance. This led to the application of these methods to secondary care administrative data in the development of a surveillance system to detect high mortality rates, emergency readmission and long length of stay in 256 diagnosis groups and 120 surgical procedure groups in collaboration with a commercial company, Dr Foster. The system is now used in 70% of acute hospital trusts in England.

Following the development of the surveillance system, it became apparent that we were identifying a number of alerts each month, some of which were generated in trusts that did not have access to Dr Foster analytical tools. In 2007, we created a system, whereby each month, we notified trusts by letter (regardless of whether they were Dr Foster customers or not) if they had alerted for a subset of diagnoses and conditions at a high alarm threshold. In the letter, we are explicit in the range of possible reasons for a signal, including random variation, poor data quality or coding problems and case-mix issues, and we emphasise that we draw no conclusions as to what lies behind the figures. We worked closely with the Healthcare Commission (which was to be taken over by the Care Quality Commission (CQC)) in setting up this Mortality Alerting and Surveillance System (MASS), and agreed to notify them about the letters we sent. The CQC subsequently set up their own alerting system, based on similar methods, but their data were originally less current and based around Healthcare Resource Groups (HRGs). Within the first 8 months of running our monthly alerts, one hospital had a series of 6 mortality alerts in a range of conditions and procedures. This helped trigger an inspection of care at Stafford Hospital, which was within Mid Staffordshire NHS Foundation Trust. The initial inspection team found “appalling” failures of emergency care within the trust, and a number of inquiries, culminating in the public inquiry led by Robert Francis, confirmed a litany of failings. The Inquiry report recognised the role that our surveillance system of mortality alerts had to play in identifying Mid Staffs as an outlier. Key recommendations reflecting our unit’s work, were that all healthcare provider organisations should develop and maintain systems which give effective real-time information on the performance of each of their services, specialist teams and consultants in relation to mortality, patient safety and minimum quality standard.

The Dr Foster Unit's work in patient safety was noted in the Mid Staffordshire Public Inquiry:

• 5.237 "There is no doubt that, without the work of the Dr Foster Unit and Dr Foster Intelligence, comparative mortality statistics would not have been published as quickly, or as fully, as they now are.”
•  5.192 “There is now a consensus that significantly high HSMR/SHMI results should trigger a serious consideration of whether poor care is an explanation for them, whatever other steps are also taken. In any event, the explanation of poor coding offers little reassurance that there is no matter of concern.”

Learn more about the Mid Staffordshire Public Inquiry.