|
|
| Metagora Training Materials Guidelines for Informing Policy via Data
This manual serves as the focal point for the Metagora Training Materials. The goal of Guidelines for Informing Policy via Data is to serve as both an introduction to evidence-based policy creation for the policy-oriented professional, and as a basic explanation of the process by which data relevant to policy-making are collected and analysed, for the statistician, survey methodologist, or other data-oriented professional. As such, this document is designed with the assumption that the reader may have no experience with public policy, or may have no college-level mathematics or statistics background, or may be highly trained in both fields.
To keep the manual from becoming overly cumbersome, introductory concepts in statistics, survey methodology, and public policy are available via a separate Encyclopedia of Terms. When a technical term appears in the text of the manual, clicking on that term will yield a pop-up window with a definition of the term and further explanation. For example, if the term barplot appears in the text, clicking on it will yield a detailed explanation in a separate window. Occasionally supplementary documents will be accessible via similar links.
Each chapter of this manual is accessible via a different link below. Below each chapter link is a brief description of the concepts reviewed in the chapter.
Chapter:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
- Introduction
pages 1 | 2
printable version
Why does a policy-oriented professional need to think about data collection and analysis as a tool for policy creation? What does a survey-oriented professional or statistician need to understand about the policy-creation process in order to effectively collect or analyse data for the policy-maker's purposes? And how do both types of professionals ensure that accurate, high-quality data are interpreted well and used to inform policy? This chapter will introduce the concepts behind evidence-based policy-creation and evaluation.
top
- The Policy Process: Creating Policy within Governments and Other Types of Organisations
pages 1 |
2 |
3 |
4 |
5
printable version
This chapter will review the policy process, exploring two different models: the stages model and the streams model.
top
- The Value of Data and Indicators
pages 1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11
printable version
Indicators are an often-used but seldom understood tool. This chapter discusses what data are and how they are collected, what indicators are and how they are developed, and how the different methods for data collection, indicator-creation and statistical analysis can be used in the context of the policy process particularly in relation to policy-making in the field of democratic governance and human rights.
top
- Developing Indicators and Other Statistics from Pre-existing Data
pages 1 |
2 |
3
printable version
Data are already out there, on the United Nations web site, in the national statistical agencies, and housed within NGOs. This chapter discusses the benefits and pitfalls of using found data to develop statistics and indicators.
top
- Collecting Data – Getting Started
pages 1 |
2 |
3 |
4
printable version
If no found data exist, then data must be collected. Before decisions are made about the type of data to be collected (qualitative or quantitative, subjective or objective), and how those data will be collected (via random sample survey or another method), some basic preliminary work must be done. Research goals for the project must be decided upon, and ethical considerations must be weighed carefully. Finally, the mode of data collection must be determined. This chapter will explore all of these issues.
top
- Collecting Data – Finding the People to Interview
pages 1 |
2 |
3 |
4
printable version
This chapter contains a discussion of sampling, highlighting the reasons for using a random sample or more purposive sample, determining a target population for data collection, finding a sampling frame, types of random samples, and other types of samples. Some of the pitfalls encountered in the sampling process will also be discussed, including incomplete sampling frames, under-coverage, biased samples, homeless and institutionalised populations, and the potential for non-response. The error associated with different sampling procedures and the error introduced via non-response will also be explored.
top
- Collecting Data – Creating the Statement/Questionnaire
pages 1 |
2 |
3 |
4 |
5
printable version
Creating an instrument for the collection of data is a complex task within a single culture, and a phenomenal task when the instrument is to be used in multiple languages or cultures. This chapter reviews the questionnaire-design process, with an emphasis on multi-language and multicultural surveys. After reading this section, the policy-oriented professional will gain a better understanding of the need for thorough pre-testing of questionnaires, while the survey-oriented professional will understand the basic steps required for a high-quality questionnaire development process.
top
- Collecting Data – Logistics
pages 1 |
2 |
3 |
4 |
5 |
6
printable version
Once the data-collection instrument is designed and the people to be interviewed are selected, the real work of training interviewers and collecting the data begins. This chapter discusses aspects of the logistics of data collection, including training interviewers, supervising field workers, maintaining high quality, issues related to budgeting, and the use of incentives.
top
- Storing and Processing Data
pages 1 |
2 |
3 |
4
printable version
This chapter looks at what happens after the data are collected. Whether the data are qualitative or quantitative, they must be stored in a manner that facilitates analysis. In almost all modern data-collection processes, a computer is used to store the data. Quantitative data must be entered into a database, which involves a separate staff and the need to understand the underlying data structure. Qualitative data may be stored as-is, or "coded," via a controlled vocabulary, in order to extract quantitative elements. All of these approaches will be discussed.
top
- Analysing Data and Creating Indicators
pages 1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14
printable version
Whether data are qualitative or quantitative, they will need to be analysed before they will become useful for policy creation and monitoring. This chapter focuses on quantitative data, and discusses good practices, types of quantitative analysis, principles for the creation of statistics, and how data can be used to create simple and composite indicators.
top
- Data Presentation and the Ethical Use of Data
pages 1 |
2 |
3 |
4 |
5 |
6 |
7 |
8
printable version
Presenting data well is not just a good idea, it is a matter of ethics. This chapter discusses best practices for data presentation, gives examples of bad data presentation, and examines the use of data for unethical purposes.
top
- Evaluating the Data-collection and Analysis Process
pages 1 |
2 |
3 |
4 |
5
printable version
After the data collection is over and the analysis is done, it is important to evaluate the quality of these efforts. It is rare that a data-collection process does not encounter unexpected setbacks. This chapter will discuss how to assess a data-collection and analysis process. The concepts of quality-control will also be explored.
top
- Policy Decisions and Programmes Based on Data
pages 1 |
2 |
3 |
4 |
5
printable version
Now that the process by which indicators and data become available to the policy-oriented professional has been thoroughly discussed, this chapter will return for a more thorough review of the role that indicators and data play during the creation of policy and policy programmes.
top
- Monitoring and Evaluating Policy Programmes and Decisions
pages 1 |
2 |
3 |
4 |
5 |
6
printable version
Sometimes the purpose of a data-collection project is to allow the monitoring and/or evaluation of an existing policy. This chapter will discuss the issues specific to this type of data-collection and analysis process.
top
|
