Progress 09/01/23 to 08/31/24
Outputs
Target Audience:The target audiences of BDUFB are SNAP recipients in food insecure neighborhoods in Boston - Dorchester, East Boston,Hyde Park, Mattapan and Roxbury. These neighborhoods have some of the highest rates of food insecurity - according to 2021 BRFSS data (the most recent we have data by neighborhood)- with Roxbury at 27% , Dorchester 21%, East Boston at 26%, Hyde Park at 21% and Mattapan at 35% compared to the Boston average of 17.5% in 2021. During this reporting period, we supported 5 stores - 1 in Roxbury and 4 in East Boston, as well as navigated the onboarding process for five new stores in food insecure neighborhoods that will be joining our program in the near future. Changes/Problems:Difficulties Onboarding New Stores, Particularly Implementing Discount with New Stores' Point of Sale Systems Due to the fact that our program provides incentives via instant discount at the POS, there is significant training needed for new stores joining the program, as well as working with sales engineers from their respective POS systems so that the discount can be applied to eligible transactions. Our team has worked closely with the National Grocers Association Foundation and Fair Food Network to establish relationships with the POS providers from our new stores, and ensure that the POS system has the capabilities needed for the store to participate in the program. Difficulty Hiring a Program Manager: Delays during Y2 and the first half of Y3 in hiring a BDUFB program manager led to a delay in onboading new stores and a delay in expenditures. Both of these issues were addressed halfway through Y3 once a BDUFB Program Manager was hired with 1 FTE managing this program for the first time since the beginning of the grant period. Unexpected Outcomes: The challenges of corner stores being unable to manage the significant GusNIP data reporting needs with their limited POS system capacity continued. This is an ongoing issue that has made it challenging to reliably analyze SNAP data due to limited reporting of full SNAP sales. Providing stores support in gathering SNAP sales data is one aspect of expanding assistance to existing stores and strengthening store relations. Our five new stores all have the capacity to report SNAP and SNAP produce sales to us. Data limitations Shopper survey: The vast majority of responses (83% or 60 out of 72 total survey respondents) came from a single store located in East Boston. Because this dataset is not representative across the retail partners or communities participating in BDUFB, conclusions presented here should be taken as preliminary. Transaction data: Only one retailer, Bella's Market, provided consistent transaction data on SNAP during Year 3. One other existing store, La Union, will be able to provide SNAP sales data moving forward, starting with November 2024. Our five new stores will be able to provide SNAP data. What opportunities for training and professional development has the project provided?The new BDUFB program manager has participated in a community based participatory research training program through Harvard School of Public Health, as well as partiicpating in peer to peer learning through the nutrition incentive hub communities of practice. How have the results been disseminated to communities of interest?Our Year 3 Evaluation process included more engagement with store owners, particularly concerning survey distribution and results. What do you plan to do during the next reporting period to accomplish the goals?After losing several stores in Years 1 and 2, we evaluated our program model and worked with NIFA and NTAE on a project change to focus on onboarding larger stores that would have more foot traffic and a larger number of SNAP transactions daily. This change will enable the program to increase incentive redemptions for the remainder of the grant. We are currently onboarding five mid-sized stores to the program, and increased the maximum daily cap for customers at 6 (of the new total 9) stores from $10 to $20 per day in order to increase program efficacy and participation. Our team has worked with our new stores and their POS provider to determine how to best configure the system at each store to operate the program successfully and collect all relevant data. This will enable each store to provide data for the majority of the core metrics with their existing POS system once they began operating the program in Year 4. In addition, for one of our currently operating stores, we were able to work with the store owner and work through the process of improving access to reporting to better obtain core metrics - SNAP sales in particular -during Year 4. Through work with our existing participating stores, we have evaluated non-tech solutions to collect all core metrics, including through addressing operational, training and technical assistance needs of these stores. Operational and capacity support for BDUFB store owners and staff has been developed to both increase promotion of the program and enable them to provide a larger number of core metrics more consistently. In order to continue to improve the SNAP sales data available from stores, the Program Coordinator is meeting individually with store owners and staff to build partnerships that will allow her to support them in person and by phone with their SNAP processor/POS and other tech consultants to be able to get SNAP sales data monthly from all the stores. So far, this work has led to receiving the SNAP sales for one currently participating store that previously was not able to submit SNAP sales. In order to continue to improve the SNAP sales data available from stores, the Program Coordinator is meeting individually with store owners and staff to build partnerships that will allow her to support them in person and by phone with their SNAP processor/POS and other tech consultants to be able to get SNAP sales data monthly from all the stores. So far, this work has led to receiving the SNAP sales for one currently participating store that previously was not able to submit SNAP sales.
Impacts
What was accomplished under these goals?
Goal 1: Increase consumption of fresh fruits and vegetables among SNAP participants in neighborhoods with a high rate of food insecurity in Boston Objective 1: Increase BDUFB redemptions of fruits and vegetables 5% yearly, across BDUFB stores over 3 years. Increase SNAP sales of fruits and vegetables 5% across BDUFB stores over 3 years. 1) Major activities completed / experiments conducted; In Y3, the BDUFB program: increased the maximum daily cap for customers at 6 (of the new total 9) stores from $10 to $20 per day in order to increase program efficacy and participation. increased staffing for the program expanded with the full FTE added for a Program Manager for DUFB, contributing to 1.5 FTE for the program in total. Implemented a Community Outreach Leaders Program to increase nterest in the program from eligible community members through peer-to-peer, holistic promotion and navigation support of local and statewide nutrition incentive programs including Double Up Food Bucks. engaged more fully in social media and newsletter promotion of the program, and created updated flyers in order to promote both new and existing stores. 2) Data collected Data collected during Year 3 Evaluation includes: Monthly sales and transaction data from all stores, for BDUFB and SNAP, and shopper survey data from 72 residents. 3) Summary statistics and discussion of results In Year 3, BDUFB stores conducted $39,026 in total BDUFB transactions, and $106,904. (The SNAP transactions data come from only one store, Bella's Market, which was able to provide SNAP sales.) The value of BDUFB transactions in Year 3 decreased by 6% relative to Year 2, while the number of BDUFB transactions increased by 10% relative to Year 2. This may be due to the exit of several stores from the program in Year 2 or to changes to the COVID SNAP benefits. When looking at the four stores that have participated continuously from the beginning of the grant period to present, BDUFB transactions were more stable. With only one store reporting SNAP sales data in Year 3, SNAP sales data collection in Year 3 was too limited to provide meaningful analysis of change. 4) Key outcomes or other accomplishments realized. While BDUFB transactions have decreased overall relative to Year 1, transactions have increased for select retailers. And in Y3, two out of the four on-going stores (Bella's Market and La Sultana) saw increases in the average monthly value of BDUFB transactions compared to Year 2 - one with an increase of 36%, and the other with a 76% increase. Goal 1: Increase consumption of fresh fruits and vegetables among SNAP participants in neighborhoods with a high rate of food insecurity in Boston Objective 2: Establish BDUFB nutrition incentives at 8-10 transit-accessible stores in neighborhoods with high food insecurity rates so food insecure residents in Boston have greater access to FFV over 3 years 1) Major activities completed / experiments conducted; The program has enabled 4 stores to consistently offer a 50% discount to SNAP shoppers, which has had a significant impact, particularly in the neighborhood of East Boston, where a concentration of participating stores is located. To expand the program, we have engaged five mid-sized stores to roll out the program successfully in their stores, which will begin in Q4 of 2024. All of these stores have large fresh produce sections and are eager to begin participating and promoting the program; we are working to configure their existing more advanced POS system to operate the program successfully and collect all relevant data. 2) Data collected Data collected are monthly qualitative store reports and SNAP sales for new stores to be recruited, as well as the 72 shopper surveys. Analysis of the 72 shopper surveys is presented under Goal 2. 4) Key outcomes or other accomplishments realized. Once the 5 new stores begin operating the program in Year 4, we will reach our goal of increasing the number of stores where BDUFB is available, and we will have data for the majority of the core metrics for these 5 stores. Goal 2: Demonstrate the effectiveness of Double Up Food Bucks in increasing fruit and vegetable purchases among SNAP recipients Objective 1: Evaluate the impact of the Double Up Food Bucks program and increasing fruit and vegetable purchases among SNAP recipients over 3 years. 1) Major activities completed / experiments conducted; With the support of an evaluator, we have consistently collected survey data and analyzed core metrics from participating stores, yielding information about fruit and vegetable purchases and consumption from shoppers. 2) Data collected We collected and reported monthly sales and transaction data from all stores, and analyzed and reported program metrics. Data collected includes shopper survey data from 72 residents. 83% of survey respondents came from a single store. Because this dataset is not representative across the stores participating in BDUFB, these are preliminary conclusions. 3) Summary statistics and discussion of results 64% of shopper survey respondents report that they buy more fruits and vegetables since starting participation in BDUFB. 50% of shopper survey respondents say they shop at their store more frequently because of BDUFB. 83% of shopper survey respondents say the store they visited carries fruits and vegetables that they like (somewhat or strongly agree) 95% of BDUFB survey respondents say they have had a positive or very positive experience with BDUFB. 4) Key outcomes or other accomplishments realized The data available suggest shoppers who have used BDUFB for any period of time consume slightly more (3% more) fresh fruits and vegetables than those who have never used BDUFB. Fresh fruit and vegetable consumption was highest for shoppers who use BDUFB and who have been using the discount for six months or less. A small sample analysis shows that when looking at overall (not just fresh) fruit and vegetable consumption (which include shelf stable options such as juice, salsa, and beans), consumption is slightly higher (5%) for shoppers who have not been using the BDUFB discount. While a small sample, this may suggest that individuals who have not participated in BDUFB are able to purchase more fruits and vegetables overall than those who have utilized BDUFB, but are not purchasing as many fresh fruits and vegetables (which exclude shelf stable options) as those who have used BDUFB for any length of time.The majority of respondents to the shopper survey indicated that they were able to access a variety of fruits and vegetables that they liked, and that they purchase more fruits and vegetables since starting the program. 4) Key outcomes or other accomplishments realized. Based on the surveys conducted, participation in BDUFB is supporting access to fruits and vegetables among residents. The majority of respondents to the shopper survey indicated that they were able to access a variety of fruits and vegetables that they liked, and that they purchase more fruits and vegetables since starting the program.
Publications
|
Progress 09/01/22 to 08/31/23
Outputs
Target Audience:The target audiences of BDUFB were SNAP recipients in food insecure neighborhoods in Boston - Dorchester, East Boston, Hyde Park, Mattapan and Roxbury. These neighborhoods have some of the highest rates of food insecurity - according to 2021 BRFSS data (the most recent we have)- with Roxbury at 27% , Dorchester 21%, East Boston at 26%, Hyde Park at 21% and Mattapanat 35% compared to the Boston average of 17.5%. During this reporting period, we supported 6 stores - 1 in Roxbury, 1 in Mattapan, 4 in East Boston. Changes/Problems:Goal 1. Personnel: Continued staffing challenges have impacted the project's ability to deliver on project goals and rate of expenditure. Although we were able to hire a Program Coordinator in Year 2 to provide up to .5 time on the program, that was not enough to make major headway on the project, and the Project Director ended up being more involved as our .5 FTE was eliminated from the previous shared Program Manager role so we could hire 1 FTE Program Manager with grant funds. Unexpected Outcomes: Over the course of Y1, two stores lost their designation as SNAP retailers, and while OFJ continued partnering with these stores during Y2 to try to provide redress to the issue, only 1 got their SNAP retailer status restored and that was near the end of Y2. They have not returned to the program yet. The challenges of corner stores being unable to manage the significant GusNIP data reporting needs with their limited POS system capacity continued. This is an ongoing issue that has made it challenging to reliably analyze SNAP data due to limited reporting of full SNAP sales. Outreach/Promotion. Lack of staffing and the small number of stores prevented any new communications or promotion campaigns to launch. Once new stores are onboarded, we anticipate launching a communications campaign to promote access to the program in a sufficient number of neighborhoods to make it worthwhile to invest in. Goal 2. New approach to shopper survey data collection Following the low response rate achieved through in-person intercept surveys of shoppers in Y1, the research team implemented a new approach to data collection for shopper surveys. Flyers providing information on the survey were distributed at the point of sale at all participating stores, and respondents scanned the QR code or visited the url link to respond to the survey online. The research team submitted a protocol amendment to the IRB reflecting the change in methodology, which was approved in July 2023. Bot attack on the shopper survey and rewards distribution challenges: While the new data collection approach was successful in reaching more respondents, the survey was attacked by bots within the first weeks of data collection, and data collection was halted. We experienced delays in rewards distribution to survey participants as a result of the bot attack and technical challenges with the rewards program vendor. The research team communicated with the IRB regarding the situation, and completed an approved protocol deviation explaining the delay in rewards distribution. We are working with NTAE/GusNIP to use lessons learned from this year to develop stronger protocols and bot protections for Year 3 data collection. Data limitations: Shopper survey: The vast majority of responses to the shopper survey (74% or 62 out of 84 total survey respondents) came from a single store. We have a relatively small sample with 19 respondents using BDUFB more than 6 months, 35 using BDUFB less than 6 months, and 24 not using BDUFB at all. Because this dataset is not representative across the retail partners participating in BDUFB, conclusions should be taken as preliminary. Transaction data: Only one retailer provided consistent transaction data on SNAP during Year 2. Responding to data requests from the program has been an on-going challenge for proprietors of the small corner stores that we work with, given limitations in technical capacity and demands on their time. The team will be working with store owners to address this in the coming year. What opportunities for training and professional development has the project provided?OFJ staff have begun participating more consistently in Communities of Practice for Brick and Mortar stores and Corner Stores, which havebeen helpful. The Program Coordinator and Project Director attended the NNIN convening in June 2023, and made a number of significant contacts, ranging from other grantees, to tech companies, to NGA Foundation, and other stores. NTAE worked closely with OFJ and our evaluation consultant, MXM, to address a bot issue that came up with the survey which will be discussed in more depth in the next section. How have the results been disseminated to communities of interest?Because of the challenges of maintaining and recruiting stores to the program, sufficient data has not been collected or analyzed to be disseminated. Dissemination of the full evaluation report upon completion of the project is planned. We are also talking with another grantee, Daily Table, about how we can combine data from our two programs and potentially disseminate it together, which would demonstrate the impact of GusNIP/DUFB on all stores in the City of Boston. What do you plan to do during the next reporting period to accomplish the goals?Goal 1. Objectives 1 and 2: Staffing: We are currently in the process of hiring a Program Manager for the program, which has been underway for nearly a year, from the request to create the position from City of Boston HR, to posting the position in late spring to a rolling interview process that has taken several months and is still ongoing. Once we have more robust staffing for the project, we will be able to ensure that the project expends funds both for incentives and promotional/operational activities. Contracted services will also be deployed to support expanded promotion of the program. Building out a protocol for the Program Coordinator to support manual data collection for SNAP sales and transaction reports from certain corner stores may be pursued as an additional support strategy. Technology Solutions and Store Support: We are currently working with the two new stores we have recruited and their POS provider to determine how to best configure their POS system at each store to be able to operate the program and collect all relevant data. For the other 6 stores, we will continue looking at ways to invest in specific upgrades or non-tech solutions to collect all data components. New communication strategies and operational support are being developed to better address the needs and capacity challenges of BDUFB store owners and staff and interested store owners. This will allow OFJ to better meet store owners where they are at with regard to increasing promotion of the program and providing data like monthly SNAP sales and transaction data. We will provide greater support for stores to be able to report SNAP sales and transaction data through funding or labor support. Community Champions: A community champion pilot will be developed and implemented in Y3 to create opportunities for program participants and community members to build their leadership skills while simultaneously championing the BDUFB program with their peers and networks. The program will be piloted in one neighborhood to assess its impact and determine whether it can be scaled up in partnership with community partners. The goals of the community champion program will be to increase knowledge of and support for the program amongst community leaders, use word of mouth strategies to increase visibility for the program, better understand the impact of the program on the ground, increase on the ground support for store owners and staff, and determine changes to the program that need to be made to better meet program and community-based goals. The community champion approach will also allow OFJ to better understand and support food access goals that community members have and build community ownership over food access in the longer term, so that future strategies are rooted in community needs and plans. Communications.A communications plan targeting the neighborhood with the most existing BDUFB stores will be developed. OFJ has many neighborhood based partners and grantees who will be involved in the development and implementation of the communications and outreach plan. Goal 2: In order to continue to improve the sample sizes for survey and focus groups, OFJ will continue working closely with our evaluation consultant throughout the year to do an after action regarding the Y2 evaluation and to develop new strategies to get the word out via our communications plan to reach more shoppers. In order to continue to improve the SNAP sales data available from stores, the Program Coordinator is meeting individually with store owners and staff to build partnerships that will allow her to support them in person and by phone with their SNAP processor/POS and other tech consultants to be able to get SNAP sales data monthly from all the stores.
Impacts
What was accomplished under these goals?
Goal 1 Objective 1. Increase BDUFB redemptions of fruits and vegetables 5% yearly, across BDUFB stores over 3 years. Increase SNAP sales of fruits and vegetables 5% across BDUFB stores over 3 years. Boston Double Up Food Bucks (BDUFB) through the City of Boston Office of Food Justice provides SNAP participants with a dollar-for-dollar match of up to $5-10 per day when participants use their EBT card to purchase Fresh Fruits and Vegetables (FFV) at qualifying retail stores.From the launch of the program in September 2021 to date, the BDUFB program has generated a total of $159,889 in purchases of fruit and vegetables within Boston communities. After losing several stores in Year 1, we evaluated our program model and worked with NIFA and NTAE on a project change to focus on onboarding larger stores that would have more foot traffic and a larger number of SNAP transactions daily.This change will enable the program to more incentive redemptions for the remainder of the grant. In Year 2, staffing for the program expanded with the partial FTE added for a Program Coordinator hired in March 2023. Work to hire a full FTE Program Manager for DUFB has been ongoing for much of the past year, and our goal is to have someone in place by the beginning of 2024. This will allow us to double the percentage of FTE on this program, and help us make significant progress on expansion, promotion, and improved data collection. Data collected include: Monthly sales and transaction data from all stores, for BDUFB and SNAP, and shopper survey data from 84 residents. Based on these data, in Year 2, BDUFB stores conducted $41,320 in total BDUFB transactions, and $288,582 in total SNAP transactions. From the launch of the program in September 2021 to date, the BDUFB program has generated a total of $159,889 in purchases of fruit and vegetables within Boston communities. Though the average value of monthly BDUFB transactions across participating stores decreased by 47% compared to Year 1, and the average number of transactions decreased by 56% compared to Year 1, results are variable across retailers. This decline is in part attributable to the exit of several large stores from the program during Year 1. While BDUFB transactions have decreased overall relative to Year 1, transactions have increased for select retailers. While the average value and number of monthly BDUFB transactions relative to baseline and Year 1 overall, two stores in East Boston have seen increases in BDUFB transactions. These results suggest that some shoppers in East Boston, a particularly high need community, are achieving increased access to fruits and vegetables through the program. The four stores that have participated continuously since the grant began had more stable BDUFB transactions. Data on SNAP transactions was limited in Year 2, with only one retailer (Bella's Market) providing more than 2 months of data, which indicated a minimal change in SNAP transactions for that store. The limited data available suggests that shoppers who use the BDUFB discount for extended periods of time (more than six months) consume more fruits and vegetables than those who do not use the discount at all, or who use the discount for short periods of time. Goal 1 Objective 2.Establish BDUFB nutrition incentives at 8-10 transit-accessible stores in neighborhoods with high food insecurity rates so food insecure residents in Boston have greater access to FFV over 3 years The program has enabled 6 stores to consistently offer a 50% discount to SNAP shoppers, which has had a significant impact, particularly in the neighborhood of East Boston, where a concentration of participating stores is located. To expand the program, we have engaged several mid-sized stores regarding the program model, and understanding their interests, challenges and SNAP sales, in order to mutually determine if they will be a good fit for the program.These stores have more advanced POS systems which will support better data collection compared with corner stores. We are in process of onboarding two new stores for Q1 of 2024, both of which have recently received their SNAP authorization in the past 6 months. We have worked with the existing 6 stores to understand what has impacted any decreases in transactions and how we can better support their operations of the program, so that we don't lose any more stores. Data collected are monthly qualitative store reports and SNAP sales for new stores to be recruited, as well as the 84 shopper surveys. Analysis of the 84 shopper surveys is presented under Goal 2. Two new stores have been engaged and recruited in Year 2 to begin participating, distributing incentives and collecting data in Year 3. One is in Roxbury and one is in Dorchester, which are two of the neighborhoods with the highest rates of food insecurity and inequitable access to fresh fruits and vegetables in Boston. Both of these stores have large fresh produce sections and areeager to begin participating and promoting the program. They both have the same POS system, withmore built-in capability for incentives, as well as collection of SNAP sales, SNAP sales by type of fruit and vegetable item, and unique customer ID, data we have not been able to get from most of our other stores. Goal 2. Demonstrate the effectiveness of Double Up Food Bucks in increasing fruit and vegetable purchases among SNAP recipients. Objective 1: Evaluate the impact of the Double Up Food Bucks program and increasing fruit and vegetable purchases among SNAP recipients over 3 years. 84 residents participated in a survey that covers NTAE core metrics for SNAP shoppers. Note: The vast majority of responses to the shopper survey (74% or 62 out of 84 total survey respondents) came from a single store located in East Boston. Because this dataset is not representative across the retail partners or communities participating in BDUFB, conclusions presented here should be taken as preliminary.We collected and reported to NTAE monthly sales and transaction data from all stores, and analyzed and reported program metrics describing usage of BDUFB discount and consumption of fruits and vegetables by Boston residents, based on the surveys. The majority of BDUFB shoppers report positive experiences with the program, have access to fruits and vegetables that they like, and buy more fruits and vegetables as a result of their participation. FFV consumption is highest for shoppers who use BDUFB and specifically for those who have been using the discount for more than six months, compared with shoppers who don't use the program. Shoppers who have used BDUFB for more than six months consumed 13% more FFV than those who do not use BDUFB at all, and 9% more than those who have used BDUFB for six months or less. ?Overall fruit and vegetable consumption (including shelf stable options such as juice, salsa, fried potatoes and beans) is highest for shoppers who have been using the BDUFB discount for more than six months. These shoppers consume 9% more fruits and vegetables than those who do not use BDUFB at all, and 11% more than those who have used BDUFB for six months or less. However, we also found that shoppers who did not use BDUFB at all consumed 2% more fruits and vegetables overall than those who had been using the discount for six months or less. This small sample analysis suggests that a change in action (increased purchase of fresh fruits and vegetables) and condition (increased consumption of fresh fruits and vegetables, or nutrition status) may occur for individuals who participate in BDUFB over time, but that extended participation in the program may be an important factor.?
Publications
|
Progress 09/01/21 to 08/31/22
Outputs
Target Audience:The target audiences of Boston Double Up Food Bucks (BDUFB) were SNAP recipients in food insecure neighborhoods in Boston - Dorchester, East Boston, Hyde Park, Mattapan and Roxbury. SNAP recipients in Jamaica Plain were also served by this grant. These neighborhoods have some of the highest rates of food insecurity in Boston with Roxbury at 23%, Dorchester 18.1%, and Mattapan 18% compared to the Boston average of 15%. Two Jamaica Plain census tracts also have high rates of food insecurity - Census Tract 812 and Census Tract 11011.03 with 26.7% and 19.2% of residents experiencing food insecurity. Additionally, the median household income in Boston, MA is $71,834 while median income in Roxbury is $25,937, $43,256 in Mattapan, $47,200 in Dorchester (zip code: 02124), $51,549 in East Boston, and $65,260 in Hyde Park. These neighborhoods also have higher minority populations compared to Boston as a whole where 55% of the population are individuals of color. This is compared to 78% in Dorchester; 89% in Roxbury; 94% in Mattapan; 74% in Hyde Park; and 68% East Boston. During this reporting period, we supported 8 stores, 1 in Roxbury, 1 in Mattapan, 4 in East Boston and 1 in Jamaica Plain that abutted census tract 812. Changes/Problems:Goal 1: Increase consumption of fresh fruits and vegetables among SNAP participants in neighborhoods with a high rate of food insecurity in Boston Objective 1: Increase SNAP sales and BDUFB redemptions of fruits and vegetables 5% yearly, across BDUFB stores over 3 years Objective 2: Establish BDUFB nutrition incentives at 15-18 transit-accessible grocery stores in neighborhoods with high food insecurity rates so food insecure residents in Boston have greater access to FFV over 3 years Major problems or delays that may have a significant impact on the rate of expenditure: Staff personnel. A number of staffing challenges have impacted the project's ability to deliver on project goals and rate of expenditure. Initial proposed staff time on the project was not realistic to meet the operational needs of the project. More than 1.2 FTE will be needed moving forward to achieve the project's goals. This is being addressed in Y2 and Y3 through a combination of staffing and contracted services OFJ experienced significant staff turn-over over the course of Y1 due to Mayoral administration and office transitions. In addition to an overall decrease of over 50% in staff capacity, several specific staff transitions had a major impact on this project: The office director's resignation less than halfway through Y1 left the office without leadership to assess needed pivots or changes to program activities in order to better meet program goals, as a search for a new director was being conducted. The Communications Director and the Program Coordinator were two of the project's major committed staff and left at the beginning of Y1. Neither position has been replaced due to hiring challenges related to the Program Coordinator's designation as a temporary grant-funded role, but the City expects to hire a new Program Coordinator in Q1 of calendar year 2023. Unexpected Outcomes: The major decrease in number of participating stores and number of transactions resulted in large part to unanticipated changes to stores designations outside of the control of OFJ or the stores, as follows: After the grant decision was made, 2 BDUFB stores needed to be written out of this project because they were affiliated with another GusNIP grant and couldn't receive federal funding for two projects. These two stores had the highest number of monthly BDUFB transactions and sales volumes; this project's goals were based on estimates that OFJ had made including both of these two stores. Additionally, at the onset of the grant, four stores out of the original thirteen dropped out due to inability to meet data reporting. This left OFJ with 7 of the stores that were written into the initial proposal at the start of Y1. Over the course of Y1, three stores lost their designation as SNAP retailers which were either currently participating or in the process of onboarding at the time; OFJ is currently working to advocate on behalf of these stores to address the impact of this loss of retailer licenses on neighborhood food access. The results of these changes were that OFJ's baseline number of participating firms and average monthly transactions was much lower than the proposal had anticipated, and efforts to adjust for these disruptions are underway now. Finally, an additional factor that impacted store participation was POS system incompatibility and capacity for small retailers and corner stores to adapt to new POS technology. This is an ongoing issue that has made it challenging to onboard corner stores that do not have the funds to buy PoS systems that meet GusNIP data reporting needs, given that grant funds are not to be used for purchasing PoS systems. Additionally, we could not reliably analyze SNAP data due to limited reporting of full SNAP sales. Outreach/Promotion. COVID-19 significantly impacted outreach and promotion during Y1. The Program Manager had limited ability to conduct outreach with various shutdowns attributed to different COVID-19 surges across the City of Boston, exacerbated the aforementioned decreased staffing capacity and need for staff to focus on emergency food distribution and funding during the pandemic. Initial outreach plans were delayed by a prolonged contract negotiation between the City of Boston and the primary contractor for advertisements on Boston public transportation. Finally, the departure of the Communications Director one month into Y1 impacted the ability to make necessary plans or pivots in program outreach and promotion. Goal 2: Demonstrate the effectiveness of Double Up Food Bucks in increasing fruit and vegetable purchases among SNAP recipients Objective 1: Evaluate the impact of the Double Up Food Bucks program and increasing fruit and vegetable purchases among SNAP recipients over 3 years. Unexpected Outcomes and deviation from research goals: A delayed IRB approval and several other factors led to a very small number of participant surveys, and key informant interviews completed. In addition, challenges in obtaining SNAP transaction data from stores has led to unreliable store transaction data. IRB. The project's IRB approval was delayed in completion by 6 months, and final approval was not granted until July 2022, only two months before the end of grant year 1. This gave our consultants less than 2 months to complete all evaluation activities. This led to challenges in the evaluator's ability to collect sufficient qualitative data from participants or store staff, and the inability to make pivots in outreach for survey administration, for instance, given the extremely compressed timeline for Year 1 data collection and analysis. Recruiting at stores for surveys. We completed 17 BDUFB surveys this year. A few factors contributed to a lower response rate than expected: The overall store size decreased from the initial proposal, and the elimination of the two mid-size grocery stores from this project substantially reduced the volume of potential survey takers. There were situational factors like weather and the pandemic which decreased the ability to recruit participants indoors or outdoors. Due to the delayed IRB approval, we had bilingual but not bicultural staff at all survey sessions which may have impacted successful recruitment. Survey length may have discouraged some respondents from completing the survey. Key informant interviews. We completed seven interviews with store staff in Year 1. In addition to the delay due to IRB approval delays, key staff from two of the stores were out of the country and unable to complete the interview before the end of Year 1. Significant deviations from research schedules We werenot able to reliably report SNAP sales analysis because many stores have not been able to report SNAP data due to their PoS system set-up. This will be addressed in year 2. What opportunities for training and professional development has the project provided?The OFJ team has met with our NTAE evaluators monthly to discuss progress of the project. NTAE provided support on survey design and dissemination as well as IRB guidance. OFJ has also worked closely with MXM, an evaluation consultant, to design surveys for BDUFB participants and to assess impact of the program holistically. OFJhas also had a small degree of consultation with a previous evaluation team from Tufts Friedman School of Nutrition Science and Policy. These activities have provided advice and indirect professional development in monitoring and evaluation for some members of the OFJ team. On recommendation from the NTAE, OFJ has had individual consultation with the Food Trust, Fair Food Network, and the National Grocers Association Foundation to gain insight on how to improve operations of BDUFB, with a particular focus on running a program at corner stores that have a range of point-of-sales systems that do not collect minimally required GusNIP data. How have the results been disseminated to communities of interest?As this is Year 1, the initial evaluation results have not yet been disseminated. Plans for dissemination are being developed within Year 2 Planning. What do you plan to do during the next reporting period to accomplish the goals?Goal 1: Increase consumption of fresh fruits and vegetables among SNAP participants in neighborhoods with a high rate of food insecurity in Boston Objective 1: Increase SNAP sales and BDUFB redemptions of fruits and vegetables 5% yearly, across BDUFB stores over 3 years Objective 2: Establish BDUFB nutrition incentives at 15-18 transit-accessible grocery stores in neighborhoods with high food insecurity rates so food insecure residents in Boston have greater access to FFV over 3 years Staffing: Staffing for the project will be increased during Y2 in order to ensure that the project expends funds both for incentives and promotional/operational activities. Specifically, a program coordinator to support DUFB directly is being hired at the time of this report. Contracted services will also be deployed to support expanded promotion of the program. Additional staffing for the office will allow existing staff to devote more time to this project. Dedicated staffing to support manual data collection for SNAP sales and transaction reports from certain corner stores may be pursued as an additional support strategy. Advisory Group: OFJ plans tolaunch an advisory group for BDUFB in Y2 to create opportunities for other city departments, community and research partners and retail experts to advise on opportunities to better serve the community through this program. Through this advisory group, internal and external partners will be involved to a greater degree in developing, implementing and evaluating strategies to achieve the program's goals. The group will meet quarterly to provide updates on the program, and create space for advisors to contribute to necessary pivots in strategy. Community Champions: A community champion pilot will be developed and implemented in Y2 to create opportunities for program participants and community members to build their leadership skills while simultaneously championing the BDUFB program with their peers and networks. The goals of the community champion program will be to increase knowledge of and support for the program amongst community leaders, use word of mouth strategies to increase visibility for the program, better understand the impact of the program on the ground, increase on the ground support for store owners and staff, and determine changes to the program that need to be made to better meet program and community-based goals. The community champion approach will also allow OFJ to better understand and support food access goals that community members have and build community ownership over food access in the longer term, so that future strategies are rooted in community needs and plans. This program will build off of outreach work done in partnership with a community based organization in Y1.The program will be piloted in one neighborhood to assess its impact and determine whether it can be scaled up in partnership with community partners. Communications. A comprehensive communications plan targeting the neighborhood with the most existing BDUFB stores will be developed. OFJ has many neighborhood based partners and grantees who will be involved in the development and implementation of the communications and outreach plan. Outreach/Recruitment of Stores. OFJ will work with other city departments to more strategically build relationships with civic leaders in target neighborhoods to help with 1) identification of stores in target neighborhoods and 2) building trust with stores such that they want to become part of the program Store Support: New communication strategies and operational support are being developed to better address the needs and capacity challenges of BDUFB store owners and staff and interested corner stores. This will allow OFJ to better meet store owners where they are at with regard to increasing promotion of the program and providing data like monthly SNAP sales and transaction data. Technology Solutions: To find effective and efficient redemption technology solutions that meet the preferences of the store owners that will meet the data collection needs OFJ has, we will continue looking at ways to invest in specific upgrades for stores that have pre-existing POS systems and are willing to adopt a new upgrade. We will provide greater support for stores to be able to report SNAP sales and transaction data through funding or labor support. Finally, we will provide increased support for the tablet application (ex. Credibles) that the program is already using to track redemptions, rather than looking to invest in a more comprehensive product. . Assessments for Continuous Improvement: In order to more quickly determine if strategies are working during Y2, OFJ will be assessing program performance assessment quarterly by examining activities, number of stores and overall program participation. Staff and Project Director will be involved in these quarterly assessments and will be a way to make adjustments in communications with store owners as well as the above strategies to address any new challenges that come up along the way. Goal 2: Demonstrate the effectiveness of Double Up Food Bucks in increasing fruit and vegetable purchases among SNAP recipients Objective 1: Evaluate the impact of the Double Up Food Bucks program and increasing fruit and vegetable purchases among SNAP recipients over 3 years. In order to improve sample sizes for survey and focus group responses during Y2, OFJ will work closely with evaluator throughout the year to: Use our community partner relationships to both promote the program and to identify opportunities for partners to engage in survey dissemination and focus group recruitment, as well as working with them to support bicultural enumerator recruitment. Similarly, we will work with willing store staff or other community partners who may be able to be trained in Human Subjects Research Protection protocols so they can be approved as survey enumerators by the IRB. We are also considering reducing the length of the survey. Finally, new strategies are being developed to create neighborhood based ad language based focus groups in year 2. We will also build out focus groups with community partners who have networks amongst the BDUFB participation demographic and can co-create focus groups that will help with both data collection and community building around food access needs. Finally, we are exploring potential opportunities to work with a different IRB which might allow for more quick pivots without charging for amendments.
Impacts
What was accomplished under these goals?
Boston Double Up Food Bucks (BDUFB) through the City of Boston Office of Food Justice provides SNAP participants with a dollar-for-dollar match of up to $5-10 per day whenparticipants use their EBT card to purchase Fresh Fruits and Vegetables (FFV) at qualifying retail stores. BDUFB Goals- 1) Increase consumption of fresh fruits and vegetables among SNAP participants in neighborhoods with a high rate of food insecurity in Boston Goal 2) Demonstrate the effectiveness of Double Up Food Bucks in increasing fruit and vegetable purchases among SNAP recipients This year, we expanded to 2 new BDUFB stores in target neighborhoods in Boston and continued partnerships with 6 stores, while OFJ partnered with a total of 8 stores. OFJ also connected 2 local farms based in the City of Boston to 3 BDUFB stores, supporting stores in providing quality, local, affordable produce. Due to two large stores' removal from this project, and challenges with consistent expansion of stores and staffing in order to adequately promote the program, BDUFB sales amounts and transactions decreased when looking at average change across stores. Specifically, average sales decreased by 30% and average transactions decreased by 17%.Due to the evaluator's delayed IRB approval, only 17 Double Up Food Bucks recipients were surveyed. Out of these 17, FFV consumption was higher for shoppers who used BDUFB for more than 6 months than those who used BDUFB for less than 6 months. While the program's overall impact is not yet clear, this preliminary analysis suggests that individuals who participate in BDUFB over time may change behavior. Goal 1. Objective 1. Major activities completed/experiments: Partnered with CBOs for outreach at 80+ events, distributed flyers and mailers, subsidized produce from local farms for corner stores Data collected monthly: Store sales and transaction data for DUFB and SNAP Summary statistics and discussion of results: BDUFB transactions have decreased since baseline. BDUFB sales amounts ($) decreased 47% when looking at average change from September 2021 to August 2022. The number of BDUFB transactions (#) decreased 54% from September 2021 to August 2022 when looking at average change. Average change accounts for the differing number of stores that were active and provided data each month. The exit of two high volume stores part way through Y1 can explain the decrease in BDUFB transactions and sales amounts. This disguises that smaller partner stores had increases in BDUFB sales and transactions since their monthly averages are lower compared to the larger stores that dropped out. Challenges with reporting SNAP data. The project had inconsistent access to SNAP sales data from each store; due to challenges with store capacity, OFJ and the evaluator only received SNAP data for 2-3 stores per month, and the majority of these were stores wholeft the program before the end of Y1. For the SNAP sales data available, there were somewhat erratic patterns in SNAP transactions and sales at these stores. In particular, there was a pattern of increased SNAP sales and decreased SNAP transactions in March 2022, and a return to around baseline since then. Therefore, a change in SNAP sales and transactions cannot explain the decrease in BDUFB sales, and we cannot draw meaningful conclusions from the data available. Though the SNAP sales & transaction data should not be relied upon to indicate any particular pattern, they are: Overall, SNAP sales increased 7% on average per store but decreased 6% when looking at the total change across all stores between September 2021 and August 2022. This may be due to missing data or to the removal of the two larger stores from the program early in Y1. The total number of SNAP transactions decreased 30% on average per store but increased 4% when looking at the overall change from September 2021 to August 2022. Key outcomes or other accomplishments: Due to IRB delay, our shopper survey data had a very small sample of respondents consisting of 7 people using BDUFB > 6 months, 7 using BDUFB < 6 months, and three not using BDUFB at all. Due to the small sample size, any conclusions should be taken as preliminary and not significant until we have more data.Nevertheless, the qualitative survey data showed that FFV consumption was higher for shoppers who use BDUFB the longest. Out of a sample size of 17, those who used BDUFB for > 6 months consumed 11% more fresh fruits and vegetables (and 20% more fruits and vegetables overall) than those who used BDUFB for < 6 months. Those who used BDUFB at all consumed 21% more fresh fruits and vegetables (and 17% more fruits and vegetables overall) than those who did not use BDUFB. Goal 1. Objective 2. Major activities: Promotion of BDUFB to store owners and/org managers by visiting 18 'healthy' corner stores prioritizing stores by neighborhood. Cashier training of new stores and twice a month check-ins. Evaluation of point of sale systems. Ongoing advertisement in store and on social media. Data collected monthly: Number of stores visited, trained and recruited Summary statistics and discussion of results: 2 stores were recruited this year in East Boston, one of our target neighborhoods. Key outcomes or other accomplishments. Recruitment and onboarding of 2 corners stores this year, Bella's Supermarket and La Union, both in East Boston. Goal 2. Objective 1. Major activities: Translation of expanded NTAE survey into 5 languages, Survey sessions, interviews, focus groups, collecting sales and transaction data Data collected monthly: Number of survey recipients, store owner interviews and focus group participants. Monthly sales and transaction data from participating stores Summary statistics and discussion of results:In Year 1 MXM, OFJ's evaluator, surveyed a total of 42 individuals including 20 SNAP shoppers, 17 of whom used DUBF. As mentioned above, the evaluator's IRB approval was not finalized until July 2022, two months before the end of Year 1, which caused a very compressed window for surveys and therefore a very small sample size. Of the 42 people who started our survey: 3 preferred not to answer 1 of the 2 screening questions 12 were screened out 6 stopped the survey between the screening questions and the questions about BDUFB use 4 did not use BDUFB; 1 of these was removed as a suspicious response 17 used BDUFB and are included in the core metrics set 79% of BDUFB survey shoppers (11/14) report that they buy more fruits and vegetables since starting participation in BDUFB. 60% of BDUFB survey respondents (9/15) say they shop at their store more frequently because of BDUFB. Store staff also note in interviews that stores get new clientele, or shoppers come more frequently Most shoppers using BDUFB said the store they visited carries fruits and vegetables of high quality; some store staff report that they are stocking fruits and vegetables more regularly 94% of BDUFB survey respondents (15 of 16) say they have had a positive experience with BDUFB; store staff report that they are happy to have happy customers. Of the 6 BDUFB shoppers that usually use a car to travel to shop for food, 50% (3) did not use a car and instead walked or took a bicycle. Key outcomes or other accomplishments: Survey respondents who used BDUFB for more than 6 months, consumed 11% more fruits and vegetables than those who used BDUFB for less than 6 months. Those who ever used BDUFB consumed 21% more FFV than those who never used BDUFB. These outcomes are based on a small sample with 7 respondents reporting using BDUFB more than 6 months and 7 using BDUFB less than 6 months, and 3 not using BDUFB at all. Conclusions should be taken as preliminary until we have more data. Store owners and staff report that the program helps their customers financially, especially during tough times like COVID and the economic recession.
Publications
|