Mobile-Assisted Data Collection for Effective Policymaking

Notes from a multi-country water, sanitation and hygiene (WASH) monitoring program

by Sridhar Vedachalam, Alec Shannon and Kellogg J Schwab
Johns Hopkins Water Institute, Johns Hopkins University Bloomberg School of Public Health (JHSPH)

IEEE Internet Initiative eNewsletter, January 2017

An estimated 2.4 billion people lack access to an improved sanitation option, while nearly 1 billion people lack access to improved water^[1]. Lack of access to adequate water, sanitation and hygiene (WASH) is a leading cause of diarrheal illness, intestinal worm infections and poor nutritional status in developing countries. The annual reporting of WASH statistics was made possible when the United Nations established the Millennium Development Goals (MDGs) in 2000 to address a series of interconnected issues including poverty, health, sanitation and education^[2]. Reports based on the MDGs have constituted some of the most important global development policy tools used to set national priorities for WASH, and the data collected as part of that effort remains the officially cited source on water and sanitation access in much of the world.

As part of the MDG mandate on WASH, a Joint Monitoring Programme (JMP), comprised of the World Health Organization (WHO) and UNICEF, was created to summarize and report statistics. The bulk of JMP country level results come from data reported by the USAID-led monitoring program Demographic and Health Surveys (DHS), supplemented by other global surveys led by agencies such as UNICEF, the World Bank and WHO, and data coming from the respective national agencies^[3]. Based on its primacy and attention to detail, DHS Surveys have remained the gold-standard in WASH data reporting. Even though the DHS Program is experimenting with mini-surveys that employ electronic data collection, its flagship survey remains a conventional paper-based survey that is inherently susceptible to data recording and transcription errors. Conventional surveys are expensive and time-consuming, so they are typically conducted infrequently and spaced several years apart. In the case of DHS Surveys, rounds of data collection are separated by 5 years, and it can take up to 2 years to publish and present results from individual rounds^[4]. As a result, governments, practitioners, and health professionals may be working with 5 - 7-year-old data while the landscape is rapidly shifting due to policies and programs already in place. Ongoing global issues such as rapid urbanization, economic and social migration, and internal displacement due to war and conflict point to why using outdated data could lead to policies that are either flawed or not effectively targeted at those in need.

One way to make the data collection-to-policymaking process shorter is through the use of mobile phone technology to rapidly collect and disseminate data. Mobile phone surveys are increasingly used to measure a range of population outcomes, with results being comparable or superior to traditional paper surveys^[5,6]. The World Bank mapped 7,500 water points across rural Liberia in less than a month by designing a mapping software called Field Level Operations Watch (FLOW)^[7]. The Sanitation Hygiene Infant Nutrition Efficacy (SHINE) trial in Zimbabwe is another recent example that estimated cluster-specific water and sanitation access using an open-source geospatial software^[8]. Until recently, the WASH sector has seen few large-scale digital data collection initiatives that span several countries. In contrast, national-level systems, such as the District Health Information Software, are now routinely used in countries around the world for health-related data collection and reporting using mobile phones^[9].

Performance Monitoring and Accountability 2020 (PMA2020), led by the Bill & Melinda Gates Institute for Population and Reproductive Health at the JHSPH, in collaboration with local research institutions in partner countries, is a large-scale mobile phone based monitoring program^[10]. Currently, PMA2020 surveys include family planning and WASH modules, and surveys are conducted at 6-month to one year intervals in households and health care facilities. Working alongside each country’s national statistics office, PMA2020 designs a sampling framework that allows for drawing representative clusters of households that is proportional to the national distribution. PMA2020 surveys are also modular, allowing for the rapid inclusion of newer topics while consistently monitoring core metrics.

Household interviews are conducted by female resident enumerators (REs), who are hired from within the communities to enable trust-building with respondents. PMA2020 uses a version of the open-source Open Data Kit (ODK) Collect app for Android phones to collect responses to single- or multiple-select questions, dates, and numerical results. The customized survey enables REs to toggle between languages in order to conduct interviews in local languages. Additionally, PMA2020 ODK questionnaires have embedded skip patterns, allowing appropriate questions to automatically appear based on previous responses. Custom constraints prevent nonsensical responses, such as selecting “none of the above” with another choice. Warning screens flag unusual responses so the RE can confirm accurate data entry. The length of survey time varies, but routinely falls between 20 and 30 minutes in length. When complete, REs submit interview data to a secure server using the cellular data network or a Wi-Fi network. Central staff download aggregate data for cleaning, de-identification, and analysis.

Figure 1. Resident enumerators getting trained in (a) India, and (b) Uganda (photo credit: PMA2020 staff)

In addition to disseminating via mobile platforms, results from each round of data collection are summarized and shared with partner institutions using a 2-page document, which is then shared with local ministries and stakeholders. Data from each country/round is also made publicly available on the PMA2020 website to allow other researchers to conduct their own analysis and test various hypotheses. An in-house data visualization tool, PMA DataLab, presents a quick snapshot of key findings and allows users to compare results across countries and over time. Observations from multiple countries are analyzed and published in peer-reviewed journals^[11,12].

Figure 2. Work-flow of PMA2020 Mobile Surveys (credit: PMA2020)

PMA2020 results on water and sanitation access are comparable to DHS Survey estimates using much smaller samples conducted within a shorter timeframe. Due to the modular nature of the PMA2020 surveys, new questions can be rapidly and inexpensively added to enhance our understanding of certain human behaviors every 6 months rather than every few years. This will prove critical in the future as the world aggressively attempts to bridge the water and sanitation access divide. An example of the modular feature of the PMA2020 surveys can be seen in a different project where surveys were adapted to detect hotspots of Schistosomiasis, a parasitic disease transmitted via freshwater snails, in Uganda. This project also highlights the flexibility of working with apps such as ODK, as the survey tool was easily adjusted to assist with bio-specimen collection and diagnosis of individual respondents.

By the time the MDGs concluded in 2015, UN member countries adopted the next set of goals titled the Sustainable Development Goals (SDGs). These goals are more inclusive and ambitious compared to the MDGs, and are targeted to be accomplished by 2030. Specific to WASH, Goal 6 of the SDGs aspires to achieve universal and equitable access to safe and affordable drinking water and end the practice of open defecation. Given the current state of affairs and the rate of progress over the past 15 years, Goal 6 sets daunting targets – not just to implement, but also to accurately monitor and assess. It is in this context that adaptive technology and nimble policymaking become crucial to achieve these targets. The PMA2020 platform demonstrates such a feedback loop is possible to construct and operate. The development of a successful framework for cooperation and promotion of shared agendas to enable robust monitoring and evaluation of the SDGs will depend not only on the availability of appropriate technology, but also on proactive decision-making by governments, private and public institutions, and local health practitioners.

Acknowledgements - This work was supported by the Bill and Melinda Gates Foundation, The Johns Hopkins Water Institute and The Osprey Foundation of Maryland.

References:

^[1] WHO/UNICEF. Progress on Sanitation and Drinking Water: 2015 Update and MDG Assessment. 2015

^[2] United Nations. United Nations Millennium Declaration, United Nations General assembly, New York, 18 September, 2000. Available at http://www.un.org/millennium/declaration/ares552e.htm

^[3] WHO/UNICEF Joint Monitoring Programme (JMP) for Water Supply and Sanitation. JMP: Definitions and Methods. Available at: http://www.wssinfo.org/definitions-methods/data-sources/. Accessed on November 14, 2016.

^[4] DHS Program. Survey Process. United States Agency for International Development. Available at: http://dhsprogram.com/What-We-Do/Survey-Process.cfm. Accessed on November 15, 2016.

^[5] Lane, S. J., Heddle, N. M., Arnold, E., & Walker, I. A review of randomized controlled trials comparing the effectiveness of hand held computers with paper methods for data collection. BMC medical informatics and decision making, 6: 23, 2006.

^[6] Zhang, S., Wu, Q., van Velthoven, M. H., Chen, L., Car, J., Rudan, I., ... & Scherpbier, R. W. Smartphone versus pen-and-paper data collection of infant feeding practices in rural China. Journal of medical Internet research, 14(5), e119, 2012.

^[7] Kazadi, Joy; Kleemeier, Elizabeth. Mobile Phones and Water Point Mapping. World Bank, Washington, DC. 2011. https://www.openknowledge.worldbank.com/handle/10986/11058 License: CC BY 3.0 Unported.

^[8] Ntozini, R., Marks, S. J., Mangwadu, G., Mbuya, M. N., Gerema, G., Mutasa, B., Julian, T.R., Schwab, K.J., Humphrey, J.H., & Chasekwa, B. Using geographic information systems and spatial analysis methods to assess household water access and sanitation coverage in the SHINE trial. Clinical Infectious Diseases, 61(suppl 7), S716-S725, 2015.

^[9] Labrique, A. B., Vasudevan, L., Kochi, E., Fabricant, R., & Mehl, G. (2013). mHealth innovations as health system strengthening tools: 12 common applications and a visual framework. Global Health, Science and Practice, 1(2), 160–171, 2013.

^[10] Zimmerman, L., OlaOlorun, F., & Radloff, S. Accelerating and improving survey implementation with mobile technology: Lessons from PMA2020 implementation in Lagos, Nigeria. African Population Studies, 29(1), 1699, 2015.

^[11] Vedachalam, S., MacDonald, L.H., Omoluabi, E., OlaOlorun, F., Otupiri, E., Schwab, K.J. The role of packaged water in meeting global targets on improved water access. In review, 2016.

^[12] Vedachalam, S., MacDonald, L.H., Shiferaw, S., Seme, A., Schwab, K.J. Underreporting of high-risk water and sanitation practices undermines progress on global targets. In review, 2016.

---

Sridhar Vedachalam

Dr. Sridhar Vedachalam is a Postdoctoral Fellow at the Johns Hopkins Water Institute at Johns Hopkins University Bloomberg School of Public Health. In his current role, he analyzes large-scale datasets from household surveys on water, sanitation and hygiene access for PMA2020’s program countries in Africa and Asia. Prior to this position, he was at the New York State Water Resources Institute at Cornell University, where he investigated policy options for affordable water and wastewater access in the U.S., and consulted for national and state lawmakers and policy officials.

Dr. Vedachalam’s research is centered on water and wastewater infrastructure issues including assessment, planning, financing, use, and regulation. He is particularly interested in understanding how individuals and municipalities make decisions on water issues. In his role as an Editor at the Global Water Forum, he is passionate about communicating water issues to varied audiences. He received his PhD in Environmental Science and MS degrees in Environmental Economics and Mechanical Engineering, all from The Ohio State University.

Alec Shannon

Alec Shannon is PMA2020's WASH Program Specialist. She provides programmatic support and technical assistance throughout training and data collection, with a particular focus on the project’s water, sanitation, and hygiene modules. Alec has a Master of Science in Global Health from Duke University’s Global Health Institute and a Certificate in International Development Policy from Duke’s Sanford School of Public Policy. Prior to PMA2020, Alec worked for The Water Institute at UNC, where she supported projects on Community-led Total Sanitation and Household Water Treatment and Safe Storage. She has a Bachelor of Science in Environmental Technology and Management from NC State University.

Kellogg Schwab

Dr. Kellogg Schwab is a Professor in the Department of Environmental Health and Engineering at the Johns Hopkins University Bloomberg School of Public Health and a Fellow in the American Academy of Microbiology. His research focuses on environmental microbiology and engineering with an emphasis on the fate and transport of pathogenic microorganisms in water, food and the environment. This work includes extensive laboratory-based research designed to develop and evaluate molecular detection methods with subsequent application of these methods in field-based investigations.

As Director of the JHU Water Institute, Dr. Schwab helps to integrate Hopkins researchers from public health, engineering, materials science, business, medicine, behavior, policy, and economic disciplines to address the critical nexus of water, food, and energy. The goal of this program is to achieve sustainable, scalable solutions for municipal, agricultural, and industrial water and wastewater challenges here in the United States and around the world.

Editor:

Susan Hyon Parker

Susan Hyon Parker received her B.S in Computer Science from Robert Morris University. She received her engineering education jointly through Carnegie Mellon University and Massachusetts Institute of Technology. She is currently pursuing her PhD in Nanotechnology. Presently, she is a visiting student at Oxford University.

She has been an instructor with the Carnegie Mellon Open Learning Initiative since its inception in 2001. She is an active member of the IEEE Internet Initiative, IEEE Nanotechnology, IEEE Cybersecurity, IEEE Smart Grid Power & Energy Society. She has been a contributor to the IEEE ETAP Forum and Conferences and has been reviewing papers for the National Academy of Engineering - Engineer Girl Essay Contest.

Her research interests include Sensor and Ad hoc networks, Robotics, Artificial Intelligence, Cyber Physical Systems, Vehicular Communications, Graphene, Renewable Energy, IoT and Future Internet as well as Propulsion and Aerospace Systems.