Know the problem, find the solution! Monitoring sustainable WASH service delivery: opportunities and challenges

Stef Smits and Ton Schouten

Monitoring water, sanitation, and hygiene (WASH) services is a broad topic, often understood differently by different people depending on the purposes, methods, and approaches of their monitoring initiatives. This introductory chapter provides a background to the topic. It identifies key trends and developments and the opportunities and challenges that go with them. This is followed by a presentation of the main concepts and terminologies used, and specific examples and experiences with monitoring in the WASH sub-sectors are captured in the subsequent chapters. This book has been written from the perspective of strengthening local government monitoring systems for WASH because, ultimately, these systems need to be in place to improve the sustainability of WASH service delivery. Every chapter of this book, every sub-sector dealt with and every initiative described will come back to the following question: is this contributing to strengthening monitoring systems at local government level?

Keywords: monitoring, local government, service delivery, indicators, sustainability

Introduction

Trends and developments in monitoring water, sanitation, and hygiene services

Monitoring is not new in the water, sanitation, and hygiene (WASH) sector, but the way in which it is done is changing rapidly. Bostoen and Luyendijk (2013) show how monitoring the sector has evolved over the last 50 years. Over these years, United Nations (UN) bodies and other international organizations have led global monitoring efforts. Monitoring has also become an integrated part of many WASH projects. Data collection has often been a bottleneck, limiting regular updating of information after an initial assessment. Also, much of what was labelled as monitoring stopped at the level of reporting, with little action taken as a result of the monitoring. The last decade has seen a number of trends and developments that are affecting the scope of WASH monitoring and the way in which this is done.

Monitoring access to WASH has become standard practice almost everywhere. The UN Joint Monitoring Programme (JMP) has set the standards for monitoring access to water and sanitation globally. In addition, various countries have started undertaking nationwide inventories of access to water and, to a lesser extent, sanitation facilities, referred to as water point or sanitation mapping (Pearce, 2012).

Increasingly, other service delivery indicators are also being monitored; access only tells part of the story of progress in WASH. For example, progress towards the achievement of the millennium development goals (MDGs) would be significantly lower if water quality was taken into account (Bain et al., 2012; Onda et al., 2012). Recent monitoring initiatives seek to include indicators such as water quantity, quality, and reliability, and even the performance of the service provider. This is reinforced by the need to monitor progress towards the realization of the human right to water and sanitation (De Albuquerque, 2013).

Increased attention is being paid to the monitoring of 'inputs', such as finance flows and policies and legislation for WASH services. There are initiatives to monitor these at global level, through the two-yearly GLAAS (Global Analysis and Assessment of Sanitation and Drinking-Water) process, as well as at country level, for example through budget tracking.

The changes in what is being monitored are accompanied by changes in who monitors. Monitoring was often the domain of implementing organizations, reporting on numbers of new facilities built. At best, these results fed into national asset inventories, but more often they remained internal reports for funders. With a changing focus on monitoring service delivery, local and national governments in particular are getting involved, as they are ultimately responsible for delivery.

Monitoring is also getting more prominence due to the increased demand for accountability. Users of water and sanitation services seek to hold service providers to account over the services they receive. The aid effectiveness framework, as reflected in the Paris Declaration on Aid Effectiveness, highlights mutual accountability between recipient governments and donors as one of its key principles (OECD, 2005). And, as a result of a more critical attitude of taxpayers in the North with regard to the use of aid, donors seek to provide accountability for the impact of aid. Much effort has therefore gone into operationalizing the accountability relations between donors, governments, and users, for which monitoring of service delivery is a prerequisite.

Lastly, developments in information and communication technology (ICT) have significantly reduced the costs and time needed for data collection, processing, and visualization, and have provided opportunities for more stakeholders to collect and access data.

Vision

Driven by these trends, we see an emerging shared vision for the role of monitoring in the WASH sector: one where strong national sector monitoring systems enable the planning and sustainability of WASH services. Strong monitoring systems involve various elements:

• Monitoring must be engrained in the national sector institutions that have the mandate to carry out monitoring, act upon the results, and be accountable for them.

• Strong monitoring systems imply having clear institutional arrangements, with dedicated financial and human resource capacity. This also often means having arrangements to share the costs of monitoring between sector institutions and having the mechanisms to create an intrinsic motivation for carrying out that monitoring, including, for example, mandates and incentives.

• It implies having information systems, including indicator sets, surveys, and new ICT that collects and stores data.

Achieving this vision is not straightforward: it requires capitalizing on the trends and opportunities outlined above. It also means dealing with challenges such as finding a balance between the complexity of indicators and their ease and cost-effectiveness of use; and making the most of the parallel monitoring systems of national governments, projects, and international organizations.

Scope of the book

We have identified six topics of 'where the heat is' in monitoring – that is, where the trends and developments manifest themselves most strongly, and where sector stakeholders are getting to grips with the opportunities and challenges of monitoring. Together, these six topics cover the main issues in the contemporary monitoring debate.

The focus of this book is on monitoring WASH in rural areas and small towns, which have been lagging behind urban settings as far as monitoring is concerned.

In Chapter 2, on monitoring the finance, Fonseca (2014) discusses the approaches and methodologies for monitoring investments made in the sector and whether these are delivering value for money.

For monitoring to be effective, it needs to be firmly embedded in the national institutions that are mandated with service delivery. Chapter 3, by Danert (2014), elaborates on the experiences of countries that are developing their national monitoring systems.

For the next 10 to 15 years, while aid remains a major driver of WASH sector development, project monitoring is likely to be a continuing feature of the sector. In Chapter 4, Lockwood (2014) explores how project monitoring can be positive by creating innovations that can support national government systems, but often is negative due to misaligned accountability, parallel project-monitoring systems and the counterproductive use of scarce resources.

In Chapter 5, on new information and communication technology, Pearce, Dickinson, and Welle (2014) elaborate on the opportunities that ICT developments are bringing to monitoring and the factors that lie behind the successful design and implementation of ICT systems, but they also reflect critically on the problems of these innovations in monitoring service delivery.

Van der Voorden and Krukkert (2014) consolidate trends in monitoring sanitation and hygiene in Chapter 6, focusing on the complexities of monitoring community-led total sanitation, open defecation-free (ODF) status, sanitation markets, and hygiene practices, among other issues.

Global and regional monitoring efforts have made a significant contribution to national monitoring systems. However, differences in data and definitions between these levels are often a source of confusion. Cross (2014) discusses the challenges of building coherence in global, regional, and national monitoring in Chapter 7.

Conceptual framework for monitoring WASH service delivery

In this section we propose definitions and key concepts, going through the why, what, who, how, and how much questions for monitoring. While not aiming to be exhaustive, we intend to cover the most common forms of monitoring in the WASH sector, and the ways in which terminology and concepts are understood.

Why? – Purposes for monitoring

The Oxford English Dictionary defines monitoring as the observation and checking of progress or quality of 'something' over a period of time, or keeping it under systematic review. The implicit assumption behind this is that monitoring is done recurrently to see whether an expected result is achieved, and to take action if what is observed deviates from what was expected.

The purposes of monitoring WASH service delivery are manifold, as different stakeholders have different information needs. A user of a water point in Ethiopia may want to check the books of the water committee to make sure that the tariff they pay is used to maintain the pump; the water committee member may want to monitor the income from those tariffs to see whether the costs of all necessary repairs can be covered; the woreda (district) official wants to monitor which pumps in the area are non-functional so that they can send a handpump mechanic to help with repairs; a person in the Ministry of Water and Energy monitors functionality rates of handpump throughout the country to see whether national targets are met and to analyse whether the operation and maintenance framework is leading to results; a Dutch government official monitors expenditure on WASH in Ethiopia in relation to the data of the JMP to assess whether its funding to the WASH sector makes a difference; and, yet another step removed, a member of parliament in the Netherlands monitors expenditure reports of Dutch funding to WASH in Ethiopia to see whether they can explain to their constituency that Dutch tax money is being spent effectively to keep water supplies in poor countries flowing.

The list could easily be expanded beyond these examples. The number of possible purposes is almost as large as the number of stakeholders in the sector. In this book, we focus on the most common types and purposes of monitoring in the WASH sector.

• Project cycle monitoring. This refers to monitoring progress in infrastructure development projects with the purpose of achieving timely and efficient implementation of the project, according to specifications. It entails activities such as checking the quality of the construction, monitoring stocks of building materials, keeping track of expenditures and time spent on the project, and supervising contractors and builders. This type of monitoring is typically done by the entity responsible for implementation of the project (e.g. an international non-governmental organization (NGO) or contractor), but also the overseeing authority will want to know if the implementer is delivering according to the contract.

• Project or programme result monitoring. This concerns the monitoring of final outputs of the implementation, specifically in relation to the number of assets developed and the number of people who gained access to water and sanitation. Its purpose is to provide accountability for the results obtained from funds that were spent. An interesting recent development is to express results not only in terms of new WASH systems constructed or people covered, but also to include the level of services delivered in terms of water quantity, quality, reliability, and accessibility, and to monitor the strength of the enabling environment. Examples include the sustainability check used by UNICEF in Mozambique (Godfrey et al., 2009) and the proposed sustainability clause that DGIS (Directorate General for International Cooperation of the Government of the Netherlands) uses in the projects it funds (DGIS, 2012). Further details on this type of monitoring are elaborated by Lockwood in this book.

• Inventories for asset management. This refers to the regular updating of an inventory or register of all assets in an area to provide information on which systems have become dysfunctional and which ones have been repaired. Unlike the previous types of monitoring, this is not limited to a specific project or programme, but should cover an entire administrative area (e.g. a district, region, or country) and is therefore the responsibility of the relevant local or national authorities. The purpose of these inventories is one of asset management in its broadest sense: planning infrastructure development and major repairs and replacement. 'Water point mapping' is the term often referred to for the initial development of the inventory (see Pearce, 2012, for an overview; Welle, 2005; Rabbani, 2009), but regular updating of the inventory is also needed. Examples of sanitation mapping exist (Roma et al., 2012), but they are less well developed, not least because of the amount of data that would be involved (Pearce, 2012).

• Service delivery monitoring. This entails the monitoring of characteristics of the service provided (water quality, quantity, reliability, accessibility, affordability) and the performance of service providers in their roles of operation, maintenance, and administration. The purpose is to identify weaknesses or lack of compliance with national standards and norms and to define corrective action. Service delivery is often done at different levels: users monitor the service they receive on a day-to-day basis and monitor their service providers' performance; service providers typically carry out many routine monitoring tasks such as making monthly accounts of income and expenditure, or regular water quality tests; service authorities monitor the performance of the service providers in their areas, ideally against predefined service delivery indicators; and national regulators may also carry out monitoring, for example to assess whether service providers meet performance standards.

• Monitoring the enabling environment. This refers to the tracking of what AMCOW (2011) calls the service delivery pathway, or the conditions in the financial, institutional, policy, and planning environment for service delivery. The purpose is to inform decision-making processes (often at the highest policy and strategy levels) by identifying gaps and bottlenecks in the enabling environment that need to be resolved. It entails tracking whether certain policy or strategy decisions have been put into practice, but also an analysis of the impact of such decisions on actual service delivery. This type of monitoring is typically done as a joint effort by entities operating at national level, such as relevant ministries, the regulator, and development partners, sometimes in the form of joint sector reviews. It may go beyond the WASH sector and include institutions such as the Ministry of Finance. This type of monitoring has also been given new impetus by international and regional initiatives, including country status overviews (CSOs) in various countries in Africa (AMCOW, 2011), MAPAS (Monitoreo de los Avances del País en Agua Potable y Saneamiento or Monitoring of Country Progress in Drinking Water and Sanitation) in Central America, and UNICEF's Bottleneck Analysis Tool (BAT) (Hutton et al., 2013).

What? – Scope of monitoring WASH service delivery

Having described the most common purposes of monitoring, the following aspects of WASH services can be differentiated in terms of what can be monitored, following the broad service delivery pathway shown in Figure 1.1.

• Inputs are the costs, budgets, and financing of WASH services. At a global level, the GLAAS reports compile data on financial inputs from a large number of countries (WHO, 2012), while countries such as Uganda track unit costs and financing flows as part of their sector performance monitoring (Ssozi and Danert, 2012). These are complemented by studies, such as budget tracking (see, for example, Van Ginneken et al., 2012) and lifecycle cost analysis (Burr and Fonseca, 2013), that have high potential to become monitoring tools. For example, a study on costs of point source development in Mozambique by WASHCost led to a regular review to identify changes in unit costs and use them for budgeting and planning (Zita and Naafs, 2011).