introduction

In New Zealand, the District Health Board Funding and Performance Directorate (DHBF&PD) monitors the performance of public hospitals on the Minister's behalf and therefore has the responsibility for ensuring that the measures required of public hospitals adequately meet the Crown's expectations.

Back in 1999, this government body convened a working party that set itself the task to write the guidelines for a new Patient Satisfaction Survey. The 100-page report which this group produced in June 2000 outlined in great detail the methodology that all District Health Boards now are asked to implement so that they can correctly monitor patient satisfaction among their inpatient and outpatient populations. Starting with the reporting period April-June 2000, each DHB complied with the request from the HMD and sent a questionnaire to a random sample of inpatients and outpatients drawn from the monthly discharge/visit list.

The inpatient questionnaire asks 17 questions and the outpatient questionnaire asks 15 questions. The inpatient questions cover the usual topics also included in overseas Patient Satisfaction Surveys: patient perceptions of the Emergency Department, the availability of staff, the manner in which they were treated by staff (did they receive enough information, did the staff treat them with dignity and respect?), their opinion of the hospital’s facilities (safety & security, cleanliness, food), discharge procedures and the adequacy of communication between different departments involved in their care.

The outpatient questionnaire covers the usual topics such as the patients’ perceptions of the appointment system, the manner in which they were treated by staff (did they receive enough information, did staff ask permission to treat the patient?), their opinion of the clinic’s facilities (e.g., cleanliness), the adequacy of communication between different departments involved in their care and their satisfaction with the organisation of their care with other service providers.

HSCR District Health Board Surveys

While many DHBs carry out their own surveys (some, like Bay of Plenty DHB , with meticulous care, and others, like West Coast DHB, with an apparent lack of care) many DHBs outsource this job.

HSCR provides DHBs who wish to subcontract the survey to a third party with a survey analysis system that will:

* Produce regularly quarterly or monthly reports which show patient satisfaction in the entire organisation or down to Service or Speciality level
* Interrogate the database to show in what area or on what issue patient satisfaction is particularly low and thus where the greatest opportunities for improvement are
* Link survey responses to data on a Patient Management System so that additional information such as area of domicile for inpatients or speciality for outpatients can provide further insight into the type of dissatisfied patients
* Collate verbatim responses, code these and provide monthly feedback reports to ward or clinic staff allowing immediate action on processes found wanting
* Monitor improvements in patient satisfaction on specific issues and produce customised reports on specific issues such as discharge information or the adequacy of parking facilities

The New Zealand Patient Satisfaction Index

Although the District Health Board Funding and Performance Directorate has undertaken some analysis of the Patient Survey data, HSCR believes that in order to benchmark DHB performance accurately, it is necessary to weight results to allow for differences in patient population.

The reason for weighting the data lies in the need to take into account differences in age and ethnicity distributions among DHBs. For instance, it is well known that older patients are more likely to express satisfaction with services received, than are younger patients (see Fig 1. below).

Figure 1.

Thus, a DHB which has a patient profile that is much older than other DHBs may achieve a higher level of patient satisfaction, irrespective of actual differences in the quality of service received. Using weighted data will therefore increase the accuracy of the results and avoid the possibility of interpreting variances due to demographic make-up as real differences.

HSCR has thus calculated the weighting factors for each DHB on the basis of the distribution of age, gender and ethnicity and applied the weights to every record in the dataset since the Oct-Dec 2001 quarterly period. As there are approximately 14,000 records each quarter, this is quite an enormous task.

In addition to this, substantial effort has been made by HSCR staff to increase the accuracy of the dataset: in the past some DHBs used invalid codes, applied a sequence that varied from the prescribed arrangement and in some cases entire segments were missing from the dataset. In such cases HSCR was required to make contact with various DHBs to ask them to amend or even resubmit their data. However, recent improvements in the template developed by the District Health Board Funding and Performance Directorate has ensured that such omissions and inaccuracies are largely avoided.

So now, as a result of the efforts made by New Zealand District Health Boards in implementing a nation-wide Patient Survey, DHBs around New Zealand can now access a national database which presently consist of some 247,000 patient records and 3.9 million ratings from inpatients and outpatients in 21 District Health Boards

Although the New Zealand Patient Satisfaction Index is produced on a quarterly basis, most of the data analysed is actually the 12 month period. The NZPSI is available from HSCR for $500 +GST per quarterly period

GP Survey

A survey of local GPs provides DHBs or Private Health Service Providers with feedback on what GPs think about the services they supply to the primary care community. It gives DHBs an opportunity to evaluate your organisation's services thus giving you a better understanding of where you are coming short of the mark. In particular in areas where GP's have a choice as to whether they refer their patients to one of your hospitals or that of another Health Service Provider, you need to know the reasons for referral (or no referral!) so you can increase throughput and thus attract more revenue.

If you have set up interface services with the explicit objective of improving communication with primary care (e.g. a GP Newsletter, a Service Directory, CME, a Helpdesk), you need to sample GP's on their opinion regarding these services so that you know you are (or are not) meeting their needs.

Staff Surveys

A staff survey allows you to receive feedback on what staff think about working at your hospital or clinic. It provides an opportunity for staff to comment on your organisation's vision and values and planned direction or can elicit comments on proposed strategies. Staff want to be consulted and, in particular if management takes to heart the ideas and comments put forward, such consultation will produce a feeling of inclusiveness, of empowerment, that is the hallmark of many successful organisations.

Staff surveys provide hospital management and clinical leaders not only with a good indication of the general climate but also allow them to hone in on areas in which satisfaction was lower than elsewhere. Many initiatives have been developed as a direct result of information gained by such surveys, leading to increased staff satisfaction.

Annual staff surveys enable you to look back to the first baseline study and observe any change in attitude.

On-line staff surveys

HSCR has conducted many staff surveys for healthcare organisations but the most successful ones have been those completed on-line. Such web-based staff surveys have a number of advantages over paper-based postal surveys in that they:

* are much less expensive since there are no printing cost, nor postage costs, nor data input costs.
* are more accurate since data entry error rates are confined to spelling mistakes by the people completing the surveys
* are faster because as soon as the last person has completed the survey, the data can be analysed
* are more secure as validation procedures prevent unauthorised access or submission of rogue questionnaires
* allow for improved benchmarking with other organisations since questions can be asked in exactly the same way and in the same sequence

To give you an idea of what an on line survey is like please take a tour through our Trial Staff Survey. Your responses to the questions are not recorded so you can select whatever option you like.

You'll see how easy it is and, provided it is completed by a sufficient number of staff, how useful the information would be to managers and clinical leaders. This particular questionnaire provides managers and clinical leaders with information on what staff think about the organisation in terms of issues such as how well it is managed, the quality of communication, the adequacy of resources, how well performance management is implemented and how valued or appreciated they feel. This feedback enables managers and clinical leaders to identify priority areas of concern and areas for improvement. (Of course, this questionnaire is only a test questionnaire and it can be amended to meet your specific needs or replaced by your own questionnaire)