Assess, Adapt and Sustain
Step 12
The utility of the SMSS lies in its capacity to generate timely and high-quality data for accurate monitoring of health impacts at national and subnational levels. To ensure this, an assessment process should be embedded within the system’s design to routinely generate evidence on data quality. Such assessment should aim to review 1) the data collection processes, 2) the completeness and timeliness of data collected, and 3) its comparability with other data. The assessment provides an opportunity to update the population listing, thereby ensuring an accurate denominator for all key indicators. It is also an opportunity to engage communities and stakeholders. This will promote buy-in and adoption of the system, and it will facilitate the funding and resource mobilization needed to sustain the system. In this section, we briefly discuss the steps for implementing an assessment, how to adapt the system, and continue to promote its sustainability.
Key Points
For the assessment, the central question is whether the SMSS is generating data of good quality consistent with its set objectives and adequately responding to the needs of the Ministry of Health and other stakeholders. Three complementary strategies can be implemented to answer this question appropriately:
- Internal data quality assessment, including timeliness and completeness.
- External data quality assessment, comparing with other existing data sources.
- Primary retrospective data collection compared with the prospective data reported by the system.
Internal Data Quality Assessment
This component consists of a systematic assessment of trends of the monthly number of reported births and deaths. Trends should be reviewed by the date of occurrence of events and the date of reporting. The review by date of occurrence of events allows for an understanding of seasonal patterns of events and the identification of inconsistencies and outliers. The review by date of reporting allows for an assessment of whether the field data collectors are continuously and consistently reporting into the system over time. A longer term (12 months or more) retrospective review of reporting by clusters can help identify areas where data have not been consistently reported. The internal data quality review will also examine reporting by administrative region or subnational areas of interest and the effects of any known natural disasters, conflicts or other crises. In addition, the consistency in the patterns of events reported and statistics generated should be examined. For example, examining the distribution of deaths by age can help identify under-reporting of specific age groups, particularly stillbirths and neonatal deaths, as well as detect “age heaping”, meaning rounded ages are reported instead of exact ages. For verbal autopsy assessment, it is essential to compare the number of deaths for which VA interviews are conducted to the total deaths reported into the system . Finally, an internal assessment will examine the plausibility of the mortality statistics generated from the data.
External Data Assessment:
This component consists of comparing the total number of events and the statistics generated from the data to external existing data or known statistics, such as CRVS. A straightforward approach is to compare the total number of annual births and deaths expected within the system based on known estimates of crude birth and death rates, such as from a population census or national surveys such as a Demographic and Health Surveys, and the total population under surveillance in the SMSS. This comparison provides an indication of the completeness of events reporting within the system. Another layer of assessment consists in comparing all-cause and cause-specific mortality rates by age, or childhood mortality rates with estimates from recent surveys or other existing data systems that are known to produce accurate mortality estimates.
Primary Retrospective Data Collection:
Keeping track of the total population under surveillance in an SMSS can be challenging given population movement. We recommend conducting a regular population and household census every two to three years. When appropriately planned, this census can serve as an opportunity to also collect retrospective data on events in the past one or two years. Such activity must be carried out by an externally well-trained team that visits households within each cluster to collect the data. To compare the assessment data with the SMSS data at an individual event level, it is essential to collect sufficient identifying information to facilitate linkage of households, household members and events. This process can be challenging if the SMSS has not established a strong household identification system, which is often difficult.
In the case of SIS-COVE in Mozambique, the assessment data collection included a printout of all households in each cluster which data collectors carried during the fieldwork to facilitate real-time matching. The match is done either based on existing household identification or by the name of the head of the household. The community surveillance agents helped in the identification of the household during the data collection. For households successfully matched, the data collector also confirms the members of the household using a preloaded database on their tablet. The IT system established in Mozambique allows tracking households through GPS coordinates and helps monitor the completeness of the data collection. This comprehensive data collection allows obtaining an updated population by age and sex for the system but also provides an opportunity to confirm the cluster boundaries, and provides a basis for assessing the continued representativeness of the SMSS. Furthermore, representativeness of the sample can be evaluated by comparing SMSS data to an external representative data source, such as from a population survey or national household survey such as DHS.
Adapt
It is important for the SMSS to evolve and adapt in order to remain relevant to stakeholder priorities. Effective adaptation requires a clear understanding of the weaknesses of the system and the extent to which it is failing to meet goals. Key questions include whether the system adequately addresses the needs of the Ministry of Health (MoH), the CRVS system, and other stakeholders, and whether its design and sample are sufficiently robust to produce nationally and subnationally representative estimates of mortality and causes of death
Addressing these questions will require systematically collecting feedback from the MoH and stakeholders and adapting the system to collect additional data to respond to their needs. Such feedback can be collected during the dissemination of results in stakeholders’ meetings and other scientific or policy-oriented forums. The ability to respond to key stakeholders’ needs is critical not only for improving the utility of the system but for promoting the sustainability of the system by creating stronger buy-in and leveraging financial support from donors.
From a technical aspect, adaptation also involves ensuring that the system’s design and the sample remain strong and can generate representative mortality estimates with acceptable precision. Given mortality declines over time and epidemiological transitions leading to changes in cause of death patterns, the initial SMSS sample size may become insufficient for producing reliable mortality estimates in predefined domains. By collecting and analyzing the assessment data (described above), an examination of any sample distortions can be done at national and subnational levels. Based on levels of all-cause and cause-specific mortality, updated sample size calculations may be carried out to assess the size of the current SMSS sample. It may also be that over time, specific programs within the MoH or other stakeholders will desire to obtain estimates at other domains not initially included in the sample design. Responding to such demands will require a readaptation of the sample.
Sustain
Discussions of the sustainability of the SMSS must start from the initial conception and definition of the system, and it should continue throughout implementation, data integrating to CRVS, data sharing and dissemination. While securing long-term financing is key to sustainability, the system must also consistently demonstrate its utility for the country. Financial sustainability can be achieved by leveraging development partners and multilateral or bilateral donors, as well as through gradual domestic financing. Strong governance structures and clear institutional leadership are critical for development of proposals for successful fundraising. Demonstrating the utility of the SMSS demands sharing the data produced, linking/integrating within other existing data systems, particularly the CRVS, and demonstrating the quality of the estimates generated. Together, these elements reinforce stakeholder confidence and support the long-term sustainability of the system.
To Learn More
Sustainability Lessons from the Mozambique SIS-COVE Strategy
While from the outset an SMSS can be designed, financed, and implemented by country governments, often the initial years of setting up the system are externally funded with external technical assistance. Under such scenarios it is essential that a transition plan is developed to ensure full handover of the system to country leadership and ownership. We discuss below lessons learned during the transition process for Mozambique SIS-COVE.
Design the system with an eventual transition in mind: Having the intention for transition at project inception informs the choices that are made as the system is developed. For example, embedding the system with local institutions means it will be better aligned to local structures and have a more secure footing towards sustainability. Likewise, leveraging relationships with other ministries and agencies can support institutionalization upon transition.
Approach capacity building strategically for both the program’s immediate needs as well as post-transition management: This will likely require identifying and addressing local capacity needs over the life of the project.
Be careful about programmatic shortcuts that must be unwound when transition time comes: Working within local structures, especially within government, can be more complicated and takes longer to develop so shortcuts to kick off implementation promptly can be appealing; however, often those shortcuts have to be retrofitted or redesigned later.
Partners need to agree early about the specifics of transition: In particular, which aspects of the program are prioritized for transition and where adaptations will be necessary or preferred to ensure the longevity of the program. These decisions drive later considerations around funding, stakeholders, institutionalization, etc. Transition planning requires effort, strategy, and broad agreement on the goals:
- Specific attention, investment, and time should be allocated to transition processes to minimize negative consequences on program objectives.
- If transition is the goal from the outset, considerations about future funding should always be on the agenda so that both local resources as well as external ones can be fostered.
- Stakeholder management is critical to ensure current stakeholders remain committed to the program, and new or potential stakeholders can be brought on board to support the program.
Mozambique SIS-COVE’s efforts to distribute, build and restructure analytical and information technology responsibilities between partners from project inception through transition is an example that draws on all these principles.
| Last updated |
|---|
| Apr 30, 2026 |