Tools and indicators are important in evaluating the effectiveness and efficacy of palliative care interventions. In evaluating the work of the African Palliative Care Association (APCA), a number of tools have been developed, including:
- Pre- and post-test training evaluation forms;
- The APCA standards audit tool;
- Situational analysis tools;
- Interview guides;
- Focus group discussion guides;
- Observation checklists;
- Document review checklists, and;
- The APCA African Palliative Care Outcome Scale (APCA POS).
The APCA African POS was developed to measure progress in the quality of palliative care provided by services across Africa, which had previously been problematic in the absence of rigorously validated outcome instrument. Consequently, a simple and brief multi-dimensional outcome measure for palliative care (the APCA African Palliative Outcome Scale) using patient- and family-level indicators that could be employed in routine clinical practice, was developed and validated.
The APCA African POS was developed as a patient- and family-level tool to measure the outcomes of care being provided and to make recommendations on areas for improvement. It can be used within routine clinical care to enhance individual patient management, but also as a quality improvement tool and in research, and can help inform policy formation and best practices within palliative care. The APCA African POS is one of a group of palliative care outcomes scales that are being used in different contexts and settings throughout the world.
An indicator is a variable that measures an aspect of a programme at different levels of the results chain (input, output, outcome, impact). In developing and selecting good quality indicators, the researcher must determine if they can:
- Identify what will be measured to know if conditions have or have not changed;
- Provide evidence as to the achievement (or not) of results and activities;
- Be indicative, enabling one to reduce a large amount of data down to its simplest form (e.g. an indicator of organisational capacity might be the resources mobilised).
Ten criteria for assessing the quality of potential indicators include:
- Measurable: can be quantified and measured by some scale. Quantitative indicators are numerical while qualitative indicators are descriptive observations.
- Practical: data can be collected on a timely basis and at a reasonable cost. Data should be collected frequently enough to inform progress and influence the decision-making process.
- Reliable: can be measured repeatedly with precision by different actors.
- Relevant: attributable to the programme (the extent to which a result is caused by programme activities).
- Management useful: information generated is critical to decision-making.
- Direct: the indicator closely tracks the result it is intended to measure (e.g. reduced patient pain is a direct measure of the result of 'increased uptake of pain and symptom management services').
- Sensitive: serves as an early warning of changing conditions. A sensitive indicator will change proportionately and in the same direction as changes in the condition or item being measured.
- Responsive: can be changed by management control.
- Objective: the measure is operationally specific. An objective indicator has no ambiguity about what is being measured.
- Capable of being disaggregated: data can be broken down by, for example, gender, profession age, location, or some other criteria where appropriate.
In tracking national palliative care indicators, APCA has advanced a minimum dataset:
- Number of organisations provided with technical assistance (by type of technical assistance).
- Number of health service providers per country, by type, providing palliative care.
- Number, by type, of partners trained in palliative care.
- Number of national palliative care associations formed in Africa.
- Number of national palliative care associations provided with technical assistance by type of assistance.
- Percentage of palliative care patients in targeted countries who receive quality palliative care.
- Percentage of palliative care patients having access to essential palliative care drugs.
- Number of target countries adopting palliative care standards, by type.
- Number of organisations provided with technical assistance for palliative care and HIV policy development.
- Number of countries / governments with palliative care in their health strategies.
- Number of palliative care provider organisations brought together to share lessons learnt and best practices in HIV related palliative care.