How DC-Area Nonprofits Are Measuring Their Success during the Pandemic
Volunteers pack up boxes of food to be distributed to those in need at the distribution center of the Capital Area Food Bank on April 9, 2020 in Washington, DC. (Photo by Drew Angerer/Getty Images)
Millions of people have lost jobs, health insurance, and housing during the coronavirus pandemic as racial- and income-based economic and health disparities continue to grow. The Washington, DC, metropolitan region is no exception. Already one of the most economically stratified regions before the pandemic, DC’s low-income workers have lost an estimated 20,649 jobs. Structural racism’s effects on social determinants of health magnifies these disparities. The city’s Black residents face an 80 percent COVID-19 fatality rate, despite making up less than 50 percent of the city’s population.
To mitigate the impact of some of these disparities, direct service nonprofits within the DC metro region have expanded their programming and services. From housing support to food assistance to education to health care, many organizations are developing innovative remote services to meet the increasing needs of DC-area residents.
Measure4Change, an initiative to support nonprofits’ capacity to measure and evaluate their programs’ effectiveness, recently convened more than 50 DC-area nonprofit staff focused on data and learning to discuss their work during the COVID-19 pandemic. Here are a few examples of their creativity, quick thinking, and insights that could inform other nonprofits during the pandemic.
Using new survey data to help increase client supports
The Washington School for Girls, an all-scholarship, independent, Catholic day school educating girls in grades 3–8, has been using surveys to understand students’ needs during the pandemic. In the first month of the pandemic, the school sent out several short surveys to gauge students’ access to technology and work environments at home. As one measurement and evaluation team member described, “[From the survey data] we were able to mobilize pretty quickly to make sure that kids without technology had a school laptop and hot spot so they could all participate.”
The Washington School for Girls continues to use Google Forms to quickly survey students and teachers to assess how well virtual learning is working and what additional improvements they could make. Other nonprofits could similarly consider getting real-time responses to surveys using text-message, web-based, or short, in-person surveys at resource hubs or food distribution sites to adapt the services they provide.
Prioritizing the most urgent data can reduce the burden on partners and frontline staff
The Greater DC Diaper Bank (GDCDB) provides diapers, other essential baby items, and personal hygiene products to families in the DC region through a partner network. During the COVID-19 pandemic, the GDCDB expanded their distribution network by opening 20 new diaper hubs and ensuring their 46 partner organizations could scale their resource distribution practices.
At the same time, the GDCDB streamlined data collection to minimize burden on partners as they took on more work. The GDCDB tracks the number of diapers it distributes to gauge daily and weekly increases in the number of families they serve. As one staff member explains, “We ramped up quickly, adding 20 new distribution sites, and the curve was so steep, [the data show] the percentage increase in the number of diapers we’ve distributed has increased by 233 percent.” By using historic data related to service delivery and maintaining standardized practices for resource distribution, nonprofits can impute the increasing demand for services, rather than requesting additional data from partners.
Focusing on outputs over outcomes can help nonprofits better address client needs
When the pandemic forced the Latin American Youth Center (LAYC) to provide remote programming, the organization’s learning and evaluation team adapted how they monitor contact with youth participants. Now they track contact over the phone, Zoom, and social media. They also identify emerging needs of the youth they serve and note which participants they have not heard from since the onset of the pandemic.
LAYC’s tracking shows that the organization connected with 3,000 young people and their family members during the first months of the pandemic. From text messages to phone calls to video chats, the team prioritized connecting with participants during the crisis to ensure youth and their families had access critical resources, such as food.
This approach shifted LAYC’s focus from longer-term outcomes to outputs that measure LAYC’s ability to provide services responsive to COVID-19. Outcomes analysis often requires both longer-term data and significant client monitoring to assess effects. According to a learning and evaluation specialist on the team, “We don’t get to run an impact evaluation on COVID response... we’re just really output focused.… That’s the reality of solving urgent needs—you’ve just got to get people fed.” By focusing on best serving client needs, nonprofits may choose to emphasize what they can measure rather than forecasting the longer-term outcomes of this work during a time with so many unknowns.
During the COVID-19 pandemic, adaptability is key
The pandemic has driven nonprofits to adapt not only their services but also their data collection. Nonprofits’ flexibility and creativity have enabled them to expand vital services and supports for clients and update their data collection processes to prioritize clients’ new needs. Understanding how nonprofits shifted their measurement and evaluation practices can help the field respond to future crises.