CHAPTER 1
Improving programme impact and accountability through qualitative research
Abstract
Qualitative research has much to offer to the practical work of humanitarian and development organizations. Growing recognition of the potential for qualitative research to enhance programme impact is putting pressure on development practitioners to adapt a 'research approach' in their monitoring, evaluation, accountability, and learning work. This introductory chapter starts off by outlining some of the ways in which qualitative research can be used to improve the impact, quality, and accountability of development projects and programmes. It will then introduce some basic principles of qualitative research and illustrate some of the ways in which qualitative research can be incorporated into various stages of the programme cycle.
Keywords: Qualitative research; research for development; monitoring and evaluation; programme impact; programme accountability; evidence
Learning objectives
After reading this chapter, you will be able to:
• outline the ways in which qualitative research can improve development programmes and their impact;
• describe the link between qualitative research and accountability;
• explain what qualitative research is, including its strengths and weaknesses; and
• identify ways of integrating qualitative research into a programme cycle.
Key terms (definitions)
• Accountability: The means by which people and organizations are held responsible for their actions by having to account for them to other people.
• Evidence: The available body of facts or information indicating whether a belief or proposition is true or valid.
• Findings: Summaries, impressions, or conclusions reached after an examination or investigation of data.
• Formative evaluation: An early examination of an active programme with the aim of identifying areas for improvement in its design and performance.
• Generalizability: The ability to make statements and draw conclusions that can have a general application.
• Programme cycle: The process and sequence in which a programme develops from start to finish.
• Qualitative research: A method of inquiry that takes as a starting point the belief that there are benefits to exploring, unpacking, and describing social meanings and perceptions of an issue or a programme.
• Research: To study something systematically, gathering and reporting on detailed and accurate information.
Towards a 'research approach' in monitoring and evaluation
With an ever-growing emphasis on evidence-informed programming, there is a push for development practitioners to strengthen the quality of their monitoring, evaluation, accountability, and learning (MEAL) activities. For many development practitioners, evidence continues to be associated with quantitative evaluations of development initiatives. In fact, until recently, many people working in MEAL have been suspicious of qualitative methods and have had little incentive to develop a qualitative evidence capacity (Bamberger et al., 2010). While quantitative evidence is crucial for decision making and rightfully continues to play a key role in the development of evidence, there is growing recognition of the need for qualitative evidence.
This recognition is born out of the fact that development programmes have often been designed and implemented without sufficient qualitative evidence to understand the needs, wishes, and context of the target population. Too often, local perspectives have been neglected in the design, implementation, and evaluation of programmes, despite local voices containing crucial information that can help development practitioners understand pathways to programme success and failure (Chambers, 1983, 1997).
Qualitative research can systematize and formalize the process of generating qualitative evidence. Qualitative research can be used to understand the context of a programme better; it can provide us with insights to new issues and help us understand the complexity of connections and relationships between people, programmes, and organizations. It can provide beneficiaries with an opportunity to share their perspectives of an issue or a programme, which in turn can help us understand the nuances with regard to how different people experience a programme. Importantly, qualitative research can be used to ensure that development programmes resonate with local realities and expectations.
However, given the dominance of quantitative MEAL efforts, many development practitioners lack the skills and confidence to authoritatively produce qualitative evidence. In particular, there continues to be confusion and lack of clarity within development organizations about what qualitative evidence looks like and how best to conduct rigorous qualitative studies.
Although we welcome a drive for more rigorous qualitative research, we also recognize that in a 'development organization' context, there is a tension between achieving rigour, what is feasible, and what is considered useful.
We accept that some development practitioners are likely to face significant constraints in adapting some of the practices we describe in this book. We are therefore not looking to turn you into an 'academic researcher'; rather, we aim to introduce you to the 'rules of the game' for conducting rigorous qualitative research at all stages of a development project cycle. We want to encourage and equip you with the knowledge and skills required to adopt a 'research approach' (see Box 1.1) in your MEAL and development activities (Laws et al., 2013).
We believe that it is important for development practitioners to engage with qualitative research and adapt a 'research approach' in the generation of qualitative evidence for four main reasons.
• Development practitioners are at the frontline, responding to humanitarian and development needs, which makes them particularly well suited to identifying issues on which research is required and to taking an active role in facilitating research.
• Development practitioners can accelerate the use of research findings and translate them into programming and advocacy.
• With the turn to evidence-informed policy and practice, interventions need to be based on systematic qualitative research from the ground as well as on evidence from evaluations in other locations. Assessing value for money and taking programmes to scale cannot be based on anecdotes and impressions.
• Systematic qualitative research helps development practitioners improve the quality, accountability, and impact of their programmes.
This book is designed to guide development practitioners through the process of planning, conducting, and reporting on qualitative research, while simultaneously showing how qualitative methods can support the work of development practitioners. In other words, we focus on the particular uses of qualitative research in the programme cycle and highlight the role of qualitative evidence in improving the impact, quality, and accountability of development programmes.
Our practical aim is to demystify the qualitative research process and provide development practitioners with the procedural clarity, skills, and confidence to use qualitative methods authoritatively and advocate for the need to embed qualitative research in the programme cycle, either on its own or together with quantitative studies.
Qualitative research
What is qualitative research? And how is it different from quantitative research?
Research involves collecting information, also referred to as data, in a systematic way in order to answer a question. However, your research question, and the methods you use to generate data that can answer that question, are likely to reflect one of two research approaches, or a mix.
One such approach refers to quantitative research. Quantitative research typically explores questions that examine the relationship between different events, or occurrences. In an evaluation context, this might include looking at how change can be linked or attributed to a particular intervention. Such a question might be: 'What impact did child-friendly spaces have on refugee children's psycho-social well-being?' To test the causality or link between 'child-friendly spaces', an intervention, and children's 'psycho-social well-being', researchers will have to try to maintain a level of control of the different factors, also called variables, that may influence the relationship between the events. They will also need to recruit research participants randomly. Quantitative data is often gathered through surveys and questionnaires that are carefully developed, structured, and administered to provide you with numerical data that can be explored statistically and yield a result that can be generalized to some larger population (Bauer et al., 2000).
Another approach, and the focus of this book, is qualitative research. Qualitative research seeks to explore personal and social experiences, meanings, and practices as well as the role of context in shaping these. Qualitative research thus takes as a starting point the belief that there are benefits to exploring, unpacking, and describing social meanings and perceptions of a phenomenon, or a programme (Flick, 2002). Not only can qualitative research give voice to people who are ordinarily silent or whose perceptions are rarely considered, it can help explain 'how', 'why', and 'under what circumstances' does a particular phenomenon, or programme, operate as it does.
As such, you can use qualitative research to obtain information about:
• local knowledge and understanding of a given issue or programme
• people's perceptions and experiences of an issue, their needs, or a programme;
• how people act and engage with a programme, each other, and organizations;
• local responses and the acceptability and feasibility of a programme;
• meanings people attach to certain experiences, relationships, or life events;
• social processes and contextual factors (for example, social norms, values, behaviours, and cultural practices) that marginalize a group of people or have an impact on a programme;
• local agency and responses in mitigating poverty and the marginalization of vulnerable populations.
As these examples of research areas suggest, you can use qualitative research to gain a better understanding of either an issue or a particular programme or intervention. Issue-focused research can help you develop a better understanding of an issue, or phenomenon, and how it affects a group of people. This may, for example, include the health risks facing children in a particular location, or the barriers that expectant mothers face in accessing maternal healthcare. Qualitative research is particularly good at investigating sensitive topics, whether it be sexual abuse or intimate partner violence. It could also include examining the care or living arrangements of hard-to-reach groups, such as children living or working on the street. Issue-focused research can provide information that better prepares you to advocate for a cause or develop and plan a programme that addresses some of the problems that the research identifies.
Intervention studies and programme-focused research look at stakeholders' interaction with a programme. This might include looking at some of the different ways in which a programme has an impact, community-level acceptability of a programme, or the factors enabling or hindering programme success. Programme-focused research could also involve examining how beneficiaries experience a programme. For example, a research question might read: 'What are children's experiences of spending time in child-friendly spaces?' To explore children's views of 'child-friendly spaces', researchers can use creative, flexible, semi- or unstructured methods that enable and capture children's views. Such methods may include individual or group interviews (see Chapter 3), participant observations (see Chapter 4), participatory methods (see Chapter 5), or Photovoice (see Chapter 6). The information generated through these methods can be used to map out and contextualize children's social experiences or to identify a range of minority, majority, or contradictory experiences or perceptions of child-friendly spaces.
Table 1.1 summarizes key differences between qualitative and quantitative research. Although the two approaches ask different questions and have different strengths, presenting them as distinct and opposite is not overly helpful. In practice, they are often combined or draw on elements from each other (Bauer et al., 2000). For example, quantitative surveys often include open-ended questions. Similarly, qualitative responses can be quantified.
Qualitative and quantitative methods can also support each other, both through triangulation of findings and by building on each other. Triangulation is when you use different data sources and methods to shed light on an issue or programme. You can triangulate either by gathering data from different research participants or by examining an issue using different data collection methods. For example, you could compare the perspectives of teachers, students, and parents on the quality of schooling or gain an understanding of student perspectives through a questionnaire, interviews and participant observations. Why is it important to gather the perspectives of different stakeholders and/or use different methods? Triangulation can either create confidence in the trustworthiness of your findings or highlight further complexity (Denzin, 1989; Gaskell and Bauer, 2000). If, for example, different stakeholders all share a similar concern, or if your data collection methods all lead to similar observations, you are a step closer to overcoming bias (an inclination to hold a particular view) either induced by a particular method, or by only considering the views of one group of research participants. However, through data and method triangulation, you may also uncover inconsistencies or contradictions, which will require you to further understand the origin of these complexities (Gaskell and Bauer, 2000). Either way, triangulation can strengthen your conclusions and identify areas for further work.
Qualitative and quantitative methods can also be used to build on each other in an iterative manner. MEAL activities typically draw on a mix of qualitative and quantitative methods. This is because one research approach (qualitative or quantitative) can rarely fully address the research questions that are posed or provide the information required for a log frame. The approach of drawing on both qualitative and quantitative methods has been referred to as mixed methods.
The weight given to qualitative or quantitative methods may differ, as can the sequence in which qualitative and quantitative data is collected (Creswell, 2002). For example, qualitative research can be used to develop and guide the questions in a survey, and ensure that they both include relevant indicators and ask appropriate questions. Equally, a statistical analysis of a survey may identify variances, trends, and patterns, which can then be explained and explored further through qualitative research (see Figure 1.1).
