Jumping Over the Hurdles of Big Data Acquisition
This DC non-profit partnered with the USDA and Waze to provide real time information about free meals.
No Kid Hungry is a campaign of the Share Our Strength non-profit organization. The campaign seeks to increase access to federal child nutrition programs for families across the entire country.
Due to COVID-19, the team had to face unforeseen challenges, like school closures, while functioning at a faster pace than ever before.
We spoke with Emily Pia, Program Manager, and George Zoto, Senior Manager of Data Science and Engineering, to give us insight on the functionality and challenges of their texting hotline and find free meals map services.
The conversation below has been edited for length and content.
Can you describe how your team got the texting hotline and map started?
Emily: Every summer, No Kid Hungry operates a texting hotline that connects kids and families to Summer Meal Sites in their communities. This has been active for several years and has seen a lot of growth. Basically, as you text “food” (or “comida” in Spanish) to 877-877, our service asks you to send an address. After sending an address, your three closest summer meal sites will be delivered via text message to your phone.
George: We offered this service in the summer once a week, and it was just me working on the project. I got a phone call one weekend saying we had to get this running every day. Emily and others joined and the team grew to 5-7 people. We made sure the texting hotline was running while building a map where you can search for meal sites.
The challenge was data.
When we started, the USDA data wasn’t available. We had to collaborate with state agencies to get the project started. We didn’t have data at the right scale or time to make this work.
Over time, we were able to collaborate across departments and establish ongoing partnerships with multiple data providers to acquire the collection of data that we needed. By the time the USDA joined our work, we were in more than 20 states and were already connecting families with meal sites.
As this project evolved, we partnered with Waze. The challenge there was that much of the location and address data we received wasn’t standardized or formatted well. Some of them used their own versions of street names or formatted the address differently. We basically had to geocode and assign latitude/longitude for all the different sites to have standardized addresses to use on our different maps.
We used the Google Cloud Maps API to geocode thousands of sites that we found from the USDA and state agencies. The API would look up their properly formatted name and latitude/longitude. That gave us the exact address of the sites so we could visualize them and offer them to Waze to share on their apps. When their users travel and need access to emergency food, they can find a site near them.
How did your team respond to the challenges brought by COVID-19?
Emily: At the beginning of this pandemic, our team decided to reactivate this service to connect kids and families to meal sites during school closures. We started small and only worked with states we knew had data up and running. We worked with state education agencies like the Florida Department of Agriculture and Illinois Department of Public Instruction.
They collect that data statewide from the schools and non-profit organizations that are serving meals. We work directly with them to get data, either directly via email or via an API, to our database. We also worked with the USDA once they reactivated their similar service, which is in a map-form, to get data from all sides to attempt reactivating the texting service.
Since the USDA issued program flexibilities, they were changing how they served meals, where they were serving, and the days/times they served. We had to update our services quickly.
That’s less so the case now, as meals and meal service operations have reached an equilibrium and are a little more static than they were.
But we still want to make sure our data is accurate and that families are being directed to open sites that have the correct information. We’re still working through what that will look like to make sure we’re confident the data Waze is showing on their app is the most recent.
George: The moment I realized that this is now a daily service instead of a weekly service, I quickly productized a pipeline of data ingestion, aggregation, and processing so somebody else can come in and help us out.
I wrote detailed procedures on how somebody could quickly jump in after a quick tour demo to answer questions like how it works, where the data lives, and how transformation works. That’s helped us tremendously and allowed me to focus on deeper analysis of our data and new features such as adding different food sites and connecting with partners to walk through some of the data challenges we have.
What tech stack did your team use for these services?
George: I have a process in place based on Python and Tableau views to visualize and house our data, ingest and transform it, and then upload it on a public dashboard on Tableau online where anybody could see the need by state, zip code, and time.
We also used AWS for this project. With many other non-profit organizations, you probably heard of security IAM (Identity Access Management). It grants access to specific portions of the data for people to login and view. Since we didn’t have data in the beginning, we had to set up a website for users to actually share data with us.
We were able to use those services to authenticate users, send them a verification email, and do a background check. Once they were vetted, they could share data with us.
After that we used services like S3 for storing the data and Lambda and Athena for doing the backend and data processing. We used relation databases like RDS for hosting the data and doing different analysis of demand across time. We have a few other services that we used in this project like QuickSight or API Gateway. We haven’t used them here yet, but I would say there’s about 10-12 services on AWS that help us contribute to our organization’s missions.
Lastly, we used Google Cloud API for our geocoding.
What do you all anticipate focusing on in the coming months?
Emily: Other organizations that have this kind of data are going through a lot of the same challenges that we are; making sure data is validated, standardized, and exportable. I think we’re all working on the same thing at this point. So in the coming months we’ll see a lot more data sharing between us and other anti-hunger organizations.
We can all work together to best serve families and share not just where their kids can find a meal site, but where their whole family could receive that access to federal benefits and charitable food assistance programs. I’m excited to see all of that happen and we’re definitely laying a good foundation to get there.
George: We’re also considering incorporating more data sites in our DC channels – like employment information or health care information – so users can find more than just meal sites. We’re also exploring ways we can use a chat bot to customize responses for our users. Also, since some of the challenges we faced were related to big data, we’re exploring which natural language processing techniques or deep learning techniques allow us to standardize this data even further before we can serve on these challenges.
If nonprofits or individuals wanted to partner with No Kid Hungry, how would they go about doing that?
George: In the beginning, when we didn’t have enough data, we partnered with individual city agencies to give us data. Now, the main challenge I see going forward is talent and data acquisition. When we’re in a place where we need more technical expertise or guidance, we’d be willing to accept great resources to join the effort and help us with this data journey, all the way from data providing to serving families in need.