Guidelines for Informing Policy via Data

CHAPTER 3 - THE VALUE OF DATA AND INDICATORS AND THEIR ROLE IN POLICY MAKING (page 3)

3.4 RATIONALE FOR USING DATA AND INDICATORS TO ENHANCE POLICY MAKING

3.4.1 Importance of Data and Indicators in Policy making in General

Evidence-based policy-making is essential for guaranteeing that policies are designed and actions taken to respond to the needs and demands of society, and that they are not based on assumptions and beliefs. As highlighted by C. Scott, evidence-based policy-making enhances the transparency and accountability of the policy process, making it more equitable and efficient [8]. Scott enumerates several alternatives to evidence-based policy-making, such as “policy processes dictated by power and influence of sectorial interests, corruption, political ideology, arbitrariness, and anecdotal information” [9], all of which can be detrimental if they do not promote or help establish policy-making procedures that are democratic, respectful of human rights, and accountable.

Data and indicators are necessary to the policy-making process. Based on C. Scott’s paper [10], there are five main ways in which data and indicators are useful to the policy process and allow for evidence-based policy-making. They can help: 1) identify relevant issues; 2) inform the design and choice of policies; 3) forecast the future (needs and problems); 4) monitor the implementation of policies; and 5) evaluate the impact of given policies. Clear parallels can be drawn between this list and the different stages of the policy process as presented in the previous chapter (see Chapter 2 and the stages model).

Policy-making’s dependence on valid and relevant data can be illustrated using both positive and negative examples. The use of data and indicators has proven useful in cases where it has allowed for specific problems or potential dangers to be identified and addressed in a timely manner. For instance, in 2000-2001, Uganda’s Demographic and Health Survey revealed stagnant infant mortality rates in a context of overall rapid economic growth and declining poverty. The government studied this question in depth, commissioning research on the causes of this phenomenon and, after a period of analysis, took the necessary measures to tackle the root causes identified [11]. On the other hand, there are examples of cases where, because data were not available or taken into account, no actions were taken to resolve or avoid a particular problem. For instance, as C.Scott documents, “In the second half of 2001, the Malawi office of Save the Children accumulated a growing body of evidence that serious food shortages were emerging in several regions of the country.” Despite active mobilisation and firm warnings by the organisation that the country was about to suffer a major food crisis, actors on the ground paid little attention and failed to prevent and limit the inevitable famine.

Whether data and indicators are actually used or not to fulfil the above-mentioned objectives depends on two main factors: the availability of data (this refers both to their quality and quantity) and their use by policy-makers (with the assumption that policy-makers want to base their policy actions and decisions on the data, and that the data actually reaches them). The author gives four possible scenarios [12]. The first scenario, the “vicious-circle countries,” is characterised by weak data systems that are not being used. In the second, the “data supply-constrained countries,” data, although weak, are increasingly being used. The third scenario, the “data demand-constrained countries,” is characterised by improving levels of data, but those data are not being used by policy-makers. The fourth scenario, the “virtuous-circle countries,” represents the best possible situation, one where the quality and quantity of data are improving and where they are increasingly being used. Depending on which scenario applies to a given country, priority may be on obtaining solid data or on getting governments and policy-makers to use the information collected. The use of data and indicators in informing policy is thus, to a great extent, context-specific.

3.4.2 The Use of Data and Indicators in Enhancing Democratic Governance and Human Rights-Related Policy Making

The philosophy behind democratic governance and human rights-measurement exercises – and the Metagora project – is that quantifying democratic governance and human rights phenomena allows policy-makers to better understand and address those phenomena. Current human rights-monitoring mechanisms are often incomplete and only reflect specific rights. Most often, there are no data available concerning both overall democratic governance and human rights situations, and the particular processes which led to these situations. What information is available is usually based on the reporting of individual cases to human rights organisations, or on judicial decisions. Thus the information does not apply to the entire population.

An argument for quantification is that individual cases, though irreducible in their importance, fail to provide evidence of policies or patterns of human rights abuse and violations, nor do they allow for human rights to be more respected., In the context of democratic governance and human rights monitoring, precise and quantified data can help establish the numbers, frequencies, distributions, magnitude, characteristics, and patterns of human rights violations.

The belief behind the Metagora approach is that human rights, being universal and indivisible, are obviously to be fought for whenever a violation is identified, since every case must be examined; but that only quantitative information can illustrate the magnitude and trends of human rights issues, be they civil, political, social, economic or cultural rights.

Quantification is thus essential for monitoring human rights and democratic governance, though it should not be misused to classify priorities according to the numbers obtained, that is, an erroneous hierarchy should not be established based on the idea that the higher the number, the more severe the problem. On the contrary, quantification should be used to identify each problem and allow for the elaboration and proposal of efficient measures to provide redress.

Quantifying the incidence of, say, human rights abuses serves as an efficacious means of informing the debate and contributing to the elaboration of more relevant policy-making. The core strategy of the Metagora pilot project is to help create monitoring tools by gathering evidence-based information on democratic governance and human rights issues of national importance [13]. As testimony to the power of such information, the Chairperson of the National Commission on Indigenous Peoples declared, in a preface to the final report of the Metagora activity in the Philippines: “May the findings of the [project] serve as a mirror of reflection among the other duty bearers, the institutions of government and the civil society, who have yet to awaken from their slumber to fully cater to the rights of Indigenous Peoples…”

Another example is found in the Mexican activity, carried out by Fundar. Prior to Fundar’s survey, there was no credible data available as to the frequency of abuses and ill-treatment in Mexico City, thus it was impossible to say whether the problem was severe, as claimed by human rights groups, or overstated, as some law enforcement officers averred. The findings of the survey put the debate on more solid ground. The survey provides information on the nature and quality of contacts between the population and law-enforcement officers, on the presence or absence of human rights violations, on the type of contact most likely to be associated with abuses and ill-treatment, the types of abuses and ill-treatment that are most common, and which law enforcement authorities are mainly responsible for what kind of abuse.

The impact of survey findings, and other kinds of data, on policy is crucial and involves dialogue with a number of relevant actors (i.e., members of government, the media, advisers and experts, civil society) to raise awareness on current democratic governance and human rights situations, and to document the public’s perceptions as to how well their expectations have been met by the policies implemented. Further information on this issue can be found in Chapter 13.

The DIAL project in Africa is also a good example of how precise and relevant information can help design properly targeted policies to address, for example, corruption. Indeed, surveys conducted in eight West African cities demonstrated that issues perceived as overwhelming problems are not necessarily as widespread as believed, even if very real. In the case of corruption, it appeared that, though regularly experienced by a significant proportion of the population, this phenomenon is not as prevalent for the general population as experts had thought. This understanding has led to better targeted corruption-related policies.

Despite these positive results, a number of human rights defenders have opposed the quantification of human rights violations, claiming it is unethical [14]. Quantifying human rights violations and commenting on their main features and evolution does not, in any way, question the common human rights approach. On the contrary, it reaffirms that any violation is unacceptable and wholly condemnable, and that full respect of the law and of individual rights is a basic obligation of the State. Quantifying situations helps provide human rights defenders, civil society and policy-makers with precise information about violations that can help them advocate for and frame policies and strategies aimed at reducing the gap between supposed respect for human rights and the reality on the ground.

The measurement and monitoring of democratic governance and human rights can encompass various aims and objectives. Just as H.O. Sano identifies different purposes for the use of indicators in democratic governance and human rights [15], T. Landman identifies four main functions associated with human rights measurement when considering political and civil rights. These are “(1) contextual description, monitoring and documentation of violations; (2) classification of different types of violations; (3) mapping and pattern recognition of violations over time and space; and (4) secondary analysis that provides explanations for violations and policy solutions for reducing them in the future [16].” The international human rights legal framework itself contains references to the need, indeed obligation, to monitor the realisation of human rights [17].

8. Scott, C., op. cit., pp. 4-8.

9. Ibid, pp. 7-8.

10. Ibid, pp. 9-16.

11. For further information on the issue, please refer to Box 11, Scott, C, Measuring Up to the Measurement Problem. The Role of Statistics in Evidence-Based Policy-Making, London School of Economics, London, 2005, p.33.

12. Scott, C., op. cit., pp. 17-19.

13. The Metagora I pilot project covered seven activities around the world working with different partners: with the NGO Fundar, Centre for Analysis and Research in Mexico, with the Human Sciences Research Council in South Africa, with the National Commission on Human Rights in the Philippines, with DIAL and the General Secretariat of the Andean Community in several Andean countries, with DIAL in several African countries, with the Palestinian Central Bureau of Statistics in Palestine, and with the American Association for the Advancement of Science in the USA.

14. See Landman, T., "Measuring Human Rights: Principle, Practice, and Policy," Human Rights Quaterly, 26, 2004, pp. 909-910, “it can be dehumanizing to use statistics to analyse violations of human rights.”

15. Sano, H.-O., op. cit., p. 1.

16. Landman, T., op. cit., p. 909.

17. References to and requirements for human rights monitoring are included in a number of treaty provisions. For further information, see Malhotra, R., and Fasel, N., Quantitative Human Rights Indicators: A survey of major initiatives, draft for discussion, UNHCR, March 2005, p. 3.