Integrating interventions in real-world settings often has challenges. Implementation research, when conducted correctly, can enable interventions to be tested in real-life, diverse settings and contexts. Many implementation teams are eager for practical, actionable guidance as they roll out programs and evaluate impact. Drawing from our experience with the Strengthening Pharmacy Provision of Contraceptives in Kenya (Market Test) project, which focused on strengthening pharmacies’ role in providing family planning services in Kenya, we have drawn three key lessons. These recommendations are designed to help stakeholders optimize the pharmacy channel, not just for contraceptives but also as a launchpad for other innovative public health interventions.
When evaluating the effectiveness of interventions, focus on the specific elements that make them impactful in addressing identified gaps.
Often, programs move from their theories of change and actions to employing well-documented techniques to examine impact. Very few programs examine the specific aspects of the intervention, the context in which it may or may not work, and the extent to which introducing that intervention is producing the desired effect.
Implementing teams should be clear about what gaps the intervention is meant to address
Implementation teams should be clear about which intervention or suite of interventions addresses which gaps. The human-centred design process is an example of different methodologies that can help identify the gaps that must be addressed. If there is a mismatch at the beginning of any project between what gaps are to be addressed and the interventions, then measuring the effectiveness of the intervention will pose a challenge.
Case Example: Before transitioning to the implementation research phase, the Market Test Project Team deliberately outlined the existing gaps in quality service provision at the pharmacy and linked them to the interventions. This laid a clear foundation for the measurement team to design an analytical framework useful during the intervention’s effectiveness analysis at the project’s tail end. For example, in addressing the gap of low-quality family planning services in the community retail pharmacies, the team mapped their interventions to establish or improve existing safe spaces, train pharmacy providers to offer personalized quality counselling, and have in-store mentorship programs. This made it easier to collect specific data over time that would give insights on the extent to which introducing each of these interventions improved quality family planning provision in pharmacies, the specific aspects of the interventions that were beneficial to enhancing family planning, and conditions or factors that facilitated the successful adoption or introduction of these interventions.
Implementing teams should not only focus on the effectiveness of a single intervention but should consider any other contextual factors or related interventions.
Interventions rarely work in isolation. Like systems, many interconnected parts may factor into delivering the intended result.
Case Example: From the Market Test effectiveness analysis findings, an overwhelming result was that a characteristic of community pharmacies that are high performing in terms of having higher repeat clients and recording more self-care product sales, especially injectables (DMPA-SC and DMPA-IM) is that they deploy a combination of interventions such as having a hybrid safe space approach, the use of social media, the providers’ initiatives and innovation to create demand and awareness of family planning products. In such pharmacies, the providers’ and clients’ confidence builds. A second characteristic common with high-performing pharmacies was the pharmacy’s location. Within our cohort of pharmacies, those proximal to social amenities and learning institutions recorded higher family planning services. Community pharmacies serving a consistent population with a professional relationship and good client rapport are likely to perform better than those without. A fourth characteristic of high-performing pharmacies was the pharmacy provider’s initiative and drive, which was a determining factor in family planning method choice. More engaged providers were seen to have spikes in DMPA-SC self-injection uptake. These providers were seen to put effort into keeping their clients, as they would go out of their way to use TCA cards, make calls, send short text messages, or use WhatsApp to retain constant contact with them. These three examples from the Market Test demonstrate the usefulness of looking beyond a single intervention matched to a gap and illustrate the benefit of looking at other related interventions and the context in which they are being applied.
Once in agreement on the suite of solutions that can help strengthen quality family planning services, anticipate and, where possible, prepare for unintended outcomes.
While interventions are normally aimed at effecting change, it is unavoidable that unintended consequences may emerge. A best practice is to implement techniques at regular intervals to assist programs in tracking these outcomes or results and determining how they may be included in programming. Methodologies such as ripple effects mapping and others may be useful.
Case Example: inSupply Health applied a ripple effects mapping methodology to unearth the hidden stories from the pharmacy providers on their varied journeys as they provided expanded family planning methods, including injectable (DMPA–SC and DMPA–IM). Previously, the Kenyan policy could only allow the provision of contraceptive pills and barrier methods. The Market Test project used the Ripples Effects Mapping (REM) approach in three steps. First was a 30-minute peer-to-peer interview session that told the stories about the varied goals that they had at the beginning of the project, the changes that they observed as a result of the project, their motivation to keep participating in the project, and finally, what they saw as unexpected results of participating in the project. Secondly, there was a plenary session and mapping to tease out key asks from the pharmacies, and lastly, analysis sessions that helped map both intended and unintended outcomes. The results showed increased service offerings and business expansion due to layering other programs that would benefit from the infrastructure put in place to support family planning. Another unintended consequence was increased demand for long-term methods despite these pharmacies offering short-term methods. This resulted from training pharmacy providers on short- and long-term methods, besides the fact that these methods are not available in the facilities. Lessons from this exercise have helped inSupply Health communicate clearly to all stakeholders about the need to have safe spaces for counselling, which might be used for other services, and helped shape initiatives and strategies that are being rolled out to other new investments (OPC) on how to strengthen existing referral systems between the community pharmacies and the public health facilities.
Take a ‘forward-leaning’ approach to implementation, focusing on learning that can be used to scale up ‘successful’ solutions.
At the outset of most projects, key metrics aligned with their theories of change or logical frameworks typically outline key approaches, including metrics useful in conducting analysis that will inform the scaling of interventions. In real life, especially in family planning, circumstances are constantly changing. Data-related situations often change, which means that the metrics previously outlined are no longer beneficial for making decisions about what to scale, how to scale, and to what extent to scale. With this reality in mind, we suggest that you use a “forward-leaning” approach to implementation, with the primary goal of always gathering new information and insights that will help shape the process of scaling up.
Case Example: Despite having a robust implementation research framework and robust data collection processes, Key decisions were taken in small bits with a focus on what was important, what needed to stop, and what needed to continue with responses to these questions, laying the foundation for continued quality improvement. inSupply Health instituted a routine IMPACT Teams approach (link) to ensure that decisions were made based on data. inSupply Health’s Market Test investment was three-phased, with a mandatory requirement for a Stage-Gating Process to determine whether the investments would proceed to the third phase. Because of the ‘forward-leaning’ approach to implementation, the Team could earlier on, despite seeing positive indications of success, that there was increasingly detailed and strong directional data that could inform a Stage-Gating process. This was a good foundation for making a critical decision on how to proceed. As a result, an alternative that would be useful in meeting the intended goals was curated. These were having a refocused learning agenda, preparing the market for scale, and optimizing self-injection for DMPA-SC. The rapid response initiative (RRI), whose main goal was to work with a sample of community pharmacies that had participated in the program to build individualized profiles, created learning outputs that helped identify community pharmacies with lower performance against certain critical indicators for the purposes of strengthening them. On the other hand, through segmentation analysis, the Team identified clear benchmarks that distinguish pharmacies from their competitors regarding service quality, client retention, sales growth, and additional relevant directional results, explicitly looking at family planning service delivery.
Conclusion
The Market Test project demonstrates that fusing implementation strategies with research can help community pharmacies realize their full potential for providing quality family planning services. We can design smarter, scalable interventions by focusing on real-world gaps, adapting to context, and adopting continuous learning practices. These lessons provide a practical blueprint for programs aiming to improve health services through pharmacies and other channels.
