Guidelines for Informing Policy via Data

CHAPTER 3 - THE VALUE OF DATA AND INDICATORS AND THEIR ROLE IN POLICY MAKING (page 6)

3.5.3 A Presentation of Some Typologies of Democratic Governance and Human Rights Indicators

Three different typologies of indicators, which can be more or less directly associated with the main differences highlighted here, i.e., different levels and purposes of measurement, the rights to be studied, etc., are presented below.

A. First, as presented by H.O. Sano, indicators can be divided into two different kinds of instruments aimed to measure either performance or global compliance [27].

• Performance indicators are seen as referring to analyses “where assessment of policies, institutions, programmes, and business corporation performance is the central concern.”

• Global compliance indicators, on the other hand, refer to the analysis of “the global situation and changes therein,” and not to that of a specific policy or programme.

B. A second typology is suggested by M. Sudders in an attempt to go beyond the existing disagreement concerning terms and expressions and to provide a framework that reflects the most common terms. In doing so, the author divides indicators as follows [28]:

• Input/rights/commitment/de jure indicators, to measure the commitment made by countries regarding democratic governance and human rights. This could cover the inclusion, or exclusion, of principles on the respect and protection of human rights in national constitutions, the signing of international treaties, etc.

• Process/responsibility/de facto indicators, to measure whether parties are taking action to fulfil their responsibilities and commitments. This could include the existence, or non-existence, of institutions designed to ensure that the obligations are implemented.

• Output/outcome/enjoyment/performance/de facto indicators, to measure the proportion of people enjoying their rights versus those whose rights are being violated.

C. A third and alternative typology can be found in various World Bank papers that deal with the issue of building indicators to monitor poverty [29]. These authors distinguish two main types of indicators: intermediate and final indicators, each of which can be further divided. Thus, it appears that “indicators can be used to monitor progress at various stages: inputs into, outputs from policies and programmes (intermediate indicators); outcomes and impacts on households and individuals (final indicators)” [30].

• Intermediate indicators refer to indicators that measure factors that contribute to the process of achieving an outcome or impact (the focus is on policies and programmes). Such indicators can be further divided into input and output indicators, depending on the stage of the process. What is important is that inputs and outputs are not seen as objectives in themselves, but as elements that help achieve the chosen objectives, such as reducing poverty, eradicating torture, etc.
  • Input indicators measure the various financial and physical resources dedicated to achieve an objective.
  • Output indicators measure the goods and services produced by the inputs.

• Final indicators refer to indicators that measure, albeit indirectly, the effect of an intervention on an individual’s well-being and/or on the respect and protection of human rights. Such indicators can be separated into outcome and impact indicators.
  • Outcome indicators capture access to, use of, and satisfaction with public services (such as access to state-appointed lawyers, etc.). These indicators do not measure well-being and respect of human rights as such, but rather the key elements that contribute to protecting and promoting human rights.
  • Impact indicators measure key dimensions of well-being and respect for human rights such as freedom from hunger, freedom from torture, etc.

The authors provide an interesting illustration of the various types of indicators in the case of the goal of “improving child health.” Input indicators might consist of financial and physical resources, such as spending in primary health care. Output indicators could be the goods and services generated, such as the number of nurses or the availability of medicine. Outcome indicators might include access, use and satisfaction of users, and can be represented by the number of children vaccinated or the percentage within 5 km of a health centre. Impact indicators can consist of the effect on living standards and be covered by infant child mortality or the prevalence of a specific disease [31].

It is essential to monitor these different indicators, and not just one type, to understand the link between them and to identify where changes or additional efforts should be made in order to improve the human rights situation. Final indicators tend to change more slowly than intermediate ones, which should be included as much as possible to obtain swift information on a given process.

3.5.4 Further Reflections on Democratic Governance and Human Rights Indicators

The different typologies presented above focus on different aspects and priorities of measurement exercises; they do not contradict each other. In the case of the typologies presented by the World Bank and by H.O. Sano, some parallels can be drawn between performance assessments and intermediary indicators, and between global compliance and final indicators.

Indicators are not unique or irreplaceable. For instance, one right (for example, the right to a fair trial) may have several measures and indicators (such as the number of individuals who were given access to a lawyer, who were informed of the charges against them, or whose evidence and testimonies were accepted and included to the file, etc.). The use of one or another such measure will depend, among other things, on the availability of data. In cases where a specific measure is difficult to obtain, it might be useful to use proxy measures as substitute information, though this is not always acceptable [32].

In addition, though UN documents identify “possible objects of measurement (…) it does not define each right in sufficient detail to provide a basis for international measurement and comparison [33]”. The current lack of a common and universal understanding of concepts and rights creates problems for the use of indicators in the field of democratic governance and human rights; and this common understanding is only slowly making its way. Efforts to reach such an understanding and consensus include an OHCHR-based initiative. OHCHR, in consultation with a panel of experts, including committee members, special procedure mandate holders from the Human Rights Council, UN agencies, academics and non-governmental organisations, has developed a conceptual and methodological framework for identifying operationally feasible human rights indicators [34]. Building composite indicators (i.e. indicators that combine different elements into a single measure) can be difficult in the context of democratic governance and human rights issues. The formula to measure and regroup these different elements is neither easy to reach nor free from criticism. Indeed, two main methods are available: either attribute the same score to different elements or "weigh" them (i.e., give more or less importance to various elements). Neither method is perfect. When measuring respect for indigenous peoples’ rights, for example, shall all rights be given the same value or should some rights be considered as more important or vital than others? The use of composite indicators and complex indices has also been greatly questioned; they tend to be used for ranking purposes instead of focusing on assessing and evaluating the progress made by individual countries in specific areas.

Indicators are useful for measuring a given democratic governance and human rights situation and its evolution, and thus contributing to policy-making, but they are not to be used as the sole determinant in assessing given situations. Data and indicators are two of many tools available, including general knowledge of and judgment about the situation under study.

27. Sano, H.-O., op. cit., pp. 5-6.

28. Sudders, M., op. cit., p. 4.

29. See World Bank, Poverty Monitoring Guidance Note 1, Selecting Indicators, World Bank, Washington DC, 2004, Prennushi, G., Rubio, G., and Subbarao, K., Volume 1: Core Techniques and Cross-Cutting Issues, Chapter 3: Monitoring and Evaluation, World Bank, Washington DC, 2004, pp. 107-130.

30. World Bank, Poverty Monitoring Guidance Note 1, op. cit., 2004, p. 3.

31. Ibid.

32. As commented by the author, the issue around the use of proxy measures is how closely they relate to the original concept to measure. For further information, see Sudders, M., op. cit., p. 15.

33. Ibid.

34. This conceptual and methodological framework is outlined in the Report on Indicators for Monitoring Compliance with International Human Rights Instruments prepared for the Inter-Committee Meeting of the treaty bodies in June 2006 (HRI/MC/2006/7) and available in English, French and Spanish at http://www.ohchr.org/english/bodies/icm-mc/documents.htm.