How Accurate are Police and NHS Statistics?

This morning the BBC Breakfast Program said that later today the National Audit Office would raise questions about the accuracy of NHS waiting time lists while the News web pages reported similar questions about police crime statistics

I am not surprised - for many years I was a public representative on committees relating mainly to mental health in Hertfordshire and I never believed a lot of the statistics - but I knew enough to know why the local figures were likely to be only approximate and I also knew that the officials who presented the statistics also knew of the limitations - but it was necessary to present them with a straight face. [Some examples below the fold.]

Lucy

A good example was bed occupancy in mental health wards. When my daughter Lucy was a psychiatric patient many years ago the plan was to move her back into the community - and she often came home to stay. The key point was that if she started showing serious symptoms we bundled her into the car and returned her to the hospital as a matter of urgency. As when we arrived her bed was waiting for her. When this happens the term "bed occupancy" could either mean the number of beds physically slept in on a particular day, or the number of bed that were allocated to patients who might, at a few minutes notice, require to use them. Years later, looking at the statistics I discovered that some wards were reporting "beds physically occupied" at, say 90%, while other wards reported the number of "beds allocated to named patients" at 105% In fact both wards were effectively full and the useful statistics of "How many people need a bed but can't get one" was not available.

The problem is that much NHS statistics records things that are easy to count - and facts and figures on "unmet needs", etc., necessary for competent planning are not available. On one occasion (when representing patients in my home area) I was asked my opinion on whether a few thousand pounds should be saved by closing a family planning clinic. Knowing that the answer would not be available I asked how much one unwanted pregnancy cost the NHS and Social Services. Of course this figure was totally irrelevant because the only thing that really mattered was to get the final financial figure in the box at the bottom of a page correct - and if a cut in services this year meant rocketing costs next year that was next years problem.

Only two years ago I was the "victim" of a waiting list fiddle. I was due for two minor operations, and had a date for one of them. I then got a phone call saying that they had an unexpected vacancy on the other waiting list. I said it was OK by me but explained that it would mean having two operations within a week of each other. OK I was told - but when I turned up for the first pre-op appointment that was a cry of horror from the nurse examining me. There were discussions about which of the two operations was the most important - and the hospital cancelled the least urgent. And I waited and waited and waited for a new date. My GP suggested I must have been dropped from the waiting list and when I got a new appointment the consultant got rather worried when I pointed out that while this was "officially" a new appointment, in fact I had already been waiting for about double the government maximum waiting times guideline for an operation. The operation was done quickly - but I am sure the hospital statistics submitted to the government never recorded that I had been waiting some 40 weeks for an operation!

Belinda

Our family experience with the police suggests even worse problems. Just over ten years ago my daughter Belinda was assaulted by a neighbour, who was the wife of the local street bulley. She was terrified the husband would come round and fled to the local police station for help. She was assured that that the police were there to help, and she made it clear that while she was desperately worried she did not think it would help matters to bring a formal charge of assault against the neighbour. Reassured she returned home and a few days later she was asked to call at the police station. When she arrived there she discovered  they were not going to help her. Instead she was arrested and told that she had never been to the police station to report the same incident! No record had been kept of a very frightened single woman reporting an attack because she had considered that arresting her neighbour could actually make things worse. Of course recording an assault and making no arrest would look bad in the police statistics - so no record of the visit was made, although many month later (and too late to apologize to Belinda) they checked the staff rota for the night and questioned the officer who had seen her.

On being arrested Belinda became suicidal and was admitted to a psychiatric hospital. She later returned to the police station from the hospital (the hospital have denied her rights under the Mental Health Act) and on seeing her statement the police agreed that she should make a counterclaim, which she did. Unfortunately on leaving the station her bail after arrest was unintentionally renewed on a "bail after charge" form. She apparently discovered this on her return to the hospital and assumed that she had been charged behind her back - just a the police has "conveniently forgotten" her first visit to the police station - and killed herself. It may be just a co-incidence, but the coroner would not consider evidence about her treatment in the police station - and the coroner's officer was a policeman working in the police station where she had been arrested. Of course police involvement in a death would also look very bad in the police statistics ... 

It would be very easy to be critical of the individuals involved but I don't think any of the medical staff or police acted maliciously - they were simply trapped in organisations which were dominated by the need to get favourable figures into the government statistics boxes.

I know enough about statistics to know that all real world statistics is fuzzy, is dependent on the accuracy of the original records, and should never be presented without some realistic estimate of confidence limits. The real problem lies with politicians who are so pig-ignorant of statistics (or deliberately lie by ignoring the real life uncertainties in the figures) that they cherry pick exact figures which support their cause and then use them to set demanding targets which must be met or else. Because politicians set such a bad example one cannot blame those who provide our services if they run their organisations to produce the same kinds of "how to lie with statistics" results which their masters use every day.