Guidelines for Informing Policy via Data CHAPTER 2 - THE POLICY PROCESS: CREATING POLICY WITHIN GOVERNMENTS AND OTHER TYPES OF ORGANISATIONS (page 5)
2.4.3 Policy Monitoring and Evaluation
Since they are connected, policy monitoring and evaluation will be considered here as belonging to the same stage in the policy-making process. Indeed, though separate activities, they encompass the same logic and aims.
In the same way that monitoring and evaluation feed into each other, they also feed into and rely on the formulation and implementation phases. For instance, policy monitoring and evaluation depend to a great degree on the implementation phase. The monitoring of policy aims to check, for example, whether the policy and activities implemented have contributed towards the fulfilment of the initial goals and objectives of the policy and, if not, what has happened instead. This information can later be used to assess and evaluate the policy, perhaps even readjusting it if necessary.
In fact, the only way in which some kind of evaluation and assessment is feasible without the implementation phase is when it aims to study and assess not the way in which a policy was carried out, but the way in which it was formulated. This phase is, by nature, always dependent on another one.
Evaluation is not a uniform and unique process; rather, it follows different methods. E. Suchman points to five criteria that can be helpful to assess and measure the "value" of a given policy [11]. Such criteria, which will be analysed in more detail later, consist of: the effort provided, the performance obtained, the relevance and adequacy of the performance, efficiency, and the overall functioning of the process. Most authors and evaluators have added complementary criteria (such as coherence, efficacy, etc.) to this list. Special attention will be paid to the evaluation of policies in Chapter 14 of this document.
The monitoring and evaluation phase is different from that of formulation and implementation in that its activities are different in nature. Indeed, though monitoring and evaluation is, in our view, absolutely necessary to evidence-based policy-making, in contrast to policy formulation or implementation, the monitoring and evaluation phase is not vital to the policy process. It can be absent from a policy or carried out on all or part of the process, either after or while a policy is being implemented.
2.5 CONCLUDING REMARKS
The belief underlying these guidelines is that, despite the complexity of the policy process, action can be taken to influence the process, and relevant and forward-looking policies can be implemented.
Policies can be based on a long list of criteria. As mentioned by C. Scott, choices and policies are shaped by a variety of elements such as: power and the influence of vested interests, corruption, political ideology, arbitrariness, and the use of anecdotal evidence [12]. As stressed by the author, the problem with resorting to such criteria when elaborating policy is that none is “fully consistent with a democratic political process characterised by transparency and accountability” [13], nor likely to lead to equitable, efficient and effective policy outcomes. Far from leading to transparent and democratic policy-making processes, such elements commonly fail to make policy meet society’s needs and demands.
A possible and favourable alternative is evidence-based policy making. This type of decision-making relies on careful and rigorous analysis using sound and transparent data. Chapter 3 of this manual is devoted to explaining what data are and how they can be collected, and their relevance to and potential use in policy-making, particularly informed policy-making in the fields of human rights and democratic governance.
While policy-makers may know, or think they know, what they wish to achieve, they may lack sufficient and focused information on how to achieve it. In order for policy-makers to make better and better-informed decisions, it is important that they use relevant and up-to-date data. Thus, researchers, activists, non governmental organisations, governmental institutions and organisations [14], and the media can play a crucial role in the policy process, informing and fostering evidence-based policy making.
In an increasingly complex world, detailed knowledge of the policy environment is required if policies and choices are to be made that advance society’s goals and interests. In other words, as navigators of states and institutions, policy-makers need a map that tells them where they are, or should be, departing from, where they are, or should be, going, and what obstacles they might encounter en route. Evidence is crucial in providing that information.
2.6 RECOMMENDED READING
In English
Johnson, P.M., A Glossary of Political Economy Terms, Department of Political Science, Auburn, 1994-2004.
Keck, M.E., Sikkink, K., Activists Beyond Borders: Advocacy Networks in International Politics, Cornell University Press, Ithaca and London, 1998.
Porter, R.W. with Hicks, I., Knowledge Utilization and the Process of Policy Formulation: Toward a Framework for Africa, 1995, also at http://pdf.dec.org/pdf_docs/PNABX317.pdf.
Reyes, G.E., The Policy Making Process and Models for Public Policy Analysis, University of Pittsburgh, Pittsburgh, 2001.
Scott, C., Measuring up to the Measurement Problem. The Role of Statistics in Evidence-Based Policy-Making, London School of Economics, London, 2005, shorter version at http://www.paris21.org/documents/2086.pdf.
Sutton, R., The Policy Process: an Overview, Overseas Development Institute, London, 1999.
On the specific role of civil society organisations in influencing policy:
Pollard, A., and Court, J., How Civil Society Organisations Use Evidence to Influence Policy Processes: A literature review, Overseas Development Institute, London, 2005.
In French
Boussaguet, L., Jacquot, S., and Ravinet, P., eds., Dictionnaire des Politiques Publiques, Presses de la Fondation Nationale des Sciences Politiques, Paris, 2004.
Conseil Scientifique de l’Evaluation, Petit Guide de l’Evaluation des Politiques Publiques, La Documentation Française, Paris, 1996.
Lemieux, V., L’Etude des Politiques Publiques, 2e edition, Les Acteurs et leur Pouvoir, Les Presses de l’Université Laval, Canada, 2002.
Meny, Y., Thoenig, J.-C., Politiques Publiques, Imprimerie des Presses Universitaires de France, Vendôme, 1989.
Muller, P., Surel, Y., L’Analyse des Politiques Publiques, Editions Montchrestien, Paris, 1998.
11. See Lemieux, V., op.cit.
12. Scott, C., Measuring up to the Measurement Problem. The Role of Statistics in Evidence-Based Policy-Making, London School of Economics, London, 2005, pp.7-8.
13. Ibid, p. 8.
14. Governmental organisations such as national statistical offices (NSOs) can play an extremely important role in collecting data and contributing to evidence-based policy-making, as has been the case in several Metagora activities. However, such agencies and organisations need to answer a certain number of conditions, including legitimacy and efficiency, to do so. For further information, please refer to the "Lessons Learned" document created by Metagora.
| 
