Editor’s blog Wednesday 21 April 2010: Measuring up to the tailor’s motto

As the NHS scrambles frantically to pull together its flock of annual reports, the intervention in the BMJ by Lifford and Prevenost throws the issue of measuring quality nicely onto the agenda.

A friend of mine is George Borowski (he’s the ‘Guitar George’ of the Dire Straits song Sultans Of Swing). George’s dad was a tailor, and he taught his son the tailor’s motto – ‘Measure it seven times; cut it once’. This obviously dates the phrase to an era when scarcity was a fact of life. We will recognise this era very soon: we are already in it, but broadly still in denial.

There could not really be a better motto for healthcare to borrow. For all the many valid criticisms of the New Labour attempt to modernise the NHS, one unquestionable benefit has been the increased movement towards a culture that values measurement for what it tells us about performance, safety and value for money. Linking it to money, under CQUIN, will also have some impact - though care is required not to run before we can safely walk.

We’re closer than we were, but we’re not there yet. If we were there, lots of trusts would be doing weekly data runs, fortnightly reviews with executive directors and senior clinicians and monthly published activity reports (subject to correction). Board papers, and particularly annual reports, would be very easy cut-and-paste jobs.

We are not there yet.

Measuring what matters
So, do Hospital Standardised Mortality Rates (HSMR) matter to the exclusion of all other data? No, obviously not. Contrastingly, do they give us no useful information? Of course, this position is also rubbish. See the Francis Report.

How many die after treatment does matter. There are of course reasons to be outliers: trusts who treat sicker patients, or do more experimental treatments deserve recognition. Case mix adjustment captures some of this. Others, however, treat an older demographic: fine if older equals healthier, otherwise, difficult. The success of technical medicine has created new challenges.

There are also limitations, as explored in Michael Harley and Professor John Yates’ splendidly-titled paper for the Bristol InquiryExamining variation in death rates: a job for the scientist not the journalist.

Harley and Yates cited three objections to publishing league table data: inadequacies in the data used; differences between the patients selected; and variation in the level of adequacy and resources.

Yates also submitted a paper on Early identification of poor performance and major performance failure. It cites Bignall et al on seven causal features:
1. a rigidity of institutional beliefs
2. tendency to be distracted by ‘decoy problems’
3. an organisation exclusivity that disregards non-members
4. information difficulties
5. the involvement of strangers
6. failure to comply with existing regulations
7. minimizing or underestimating emergent dangers

It all sounds right, doesn’t it? Yates is overdue for rediscovery.

As to quality data, there are many sources that I would want to use. Some are soft, and probably not collected (number of spontaneous letters or cards of thanks). Others definitely exist already – crucially, the staff survey

Debating what we should measure and how is vital. Indeed, the argument that we need to measure things in healthcare has probably been won … for now.

Measurement is of most use when it is of clear relevance to the daily working life of the worker measured. It should be as timely as possible – yes, NHS IT scarcely helps, but whatever emerges from the death throes of NHS Connecting For Health (apart from some very rich commercial litigation lawyers) will have to be maximised.

Getting the data back to people frequently and in a user-friendly way also matters. My local tip – sorry, recycling centre – has a sign up; it tells you how many working days it is since the last accident on site caused loss of working time.

It would not be difficult for healthcare providers to put up some similarly interesting, thought-provoking and useful stats around the place, and it could help focus minds.



In other news
Jeremy Laurance in The Independent continues his good run with this story about a Royal College of Physicians survey over levels of cover at nighttime in hospitals, following the 48-hour European Workling Time Directive. The stand-out figure is that “Only 6 per cent of teams included a consultant on duty at night.