Manage data gaps

When the limitations analysis identifies data gaps for any metrics, the assessment team has three primary options for addressing them:

1. Seek supplemental data: If a data gap is identified, the assessment team can seek supplemental data to fill it. This could include additional secondary data sources, composite or modeled data derived from existing secondary data, or primary data collection. Guidelines for each of these potential sources are available in the seek supplemental data section. Any supplemental data must go through the same limitations analysis process as the original datasets to ensure quality and relevance.

2. Denote the metric as 'data deficient' and defer its measurement: If efforts to seek supplemental data or adjust the scope do not resolve the gap, the assessment team can denote the metric as 'data deficient' and choose to defer its measurement. While metrics labeled as 'data deficient' will not undergo validation, the assessment team must document the reason for deferring the metric within the platform. The 'data deficient' label will appear as the result for that metric on the landscape profile and report. This designation signals existing data limitations and can help highlight areas where investment, such as generating primary data, may be needed.

3. Revisit prior steps to align scope with available data: The assessment team may reconsider the scope of the assessment to better match the available data. Significant deviations from the original scope should only be made when major data gaps prevent the assessment from progressing. Such changes must be discussed with relevant stakeholders to ensure alignment with the initiative's goals. Adjustments may involve revising the selection of metrics or, if necessary, adjusting the landscape boundary to ensure that data limitations do not hinder the assessment. If no suitable data exists for an essential metric, the assessment team might propose an alternative metric that uses an available dataset to measure the same quantity. Guidelines on revisiting prior steps are available to support assessment teams in aligning the scope with the data that is accessible and relevant.

If multiple data gaps are identified, the assessment team may wish to prioritize filling them based on the available timeframe and budget. Data gaps that cannot be filled in the current assessment period may be addressed as part of a longer-term strategy for future assessments. This strategy, including planned efforts to close data gaps, may be included in the final report's conclusion, if desired.

Seek supplemental data

To address data gaps in LandScale assessments, the assessment team can pursue several strategies for acquiring supplemental data, each with specific considerations:

Additional sources of secondary data

The assessment team may intensify their search for secondary data from the types of sources available in the Data Resources Library and detailed in the select data resources section. They can also consult local subject matter experts to identify further options for filling gaps. This is especially useful for data that is already collected and readily available, which can be quickly incorporated into the assessment.

Composite or modeled data

If individual datasets are insufficient, the assessment team may combine features from multiple datasets to create a composite dataset that fills the data gap. For example, an older but more detailed ecosystem map could be updated by overwriting newly converted areas that are identified in a recent land use or land cover map. Various online datasets may enable the assessment team to generate contemporary maps of land use or land cover, ecosystems, and changes based on recent remote sensing imagery.

The assessment team may also consider the use of modeling approaches to generate data relevant to the metrics based on other input data. However, the team must have the expertise to apply these methods appropriately, and they should be cautious, as any limitations in the input data may be transferred to the model outputs.

The process for generating composite or modeled data must be clearly documented and will undergo review as part of the Step C validation.

Primary data

In the context of LandScale, primary data refers to data collected directly from within a landscape for the purpose of conducting an assessment, although it may also be gathered for other purposes. For example, the collection of primary data for use in LandScale assessments may come about through participation in collaborative monitoring or data collection efforts for the subject landscape or over larger areas, and it may address purposes and incorporate data themes that go beyond those directly relevant for LandScale.

If the need for primary data collection is identified, careful consideration must go into the resources and capacity needed to undertake what can be a long, complex, and expensive process. As such, primary data collection is often best undertaken collaboratively with other stakeholders involved in the landscape. This collaboration can help leverage resources, reduce costs, and ensure that the data collected benefits multiple parties. Primary data collection is best planned and implemented by those with expertise and experience in the disciplines and methods that pertain to the proposed data collection effort.

When using primary data in a LandScale assessment, it is essential to provide thorough documentation in the dataset’s metadata. This documentation will be reviewed as part of the Step C validation and and should include:

• Data collection methods (e.g., household survey, field sampling) and associated survey or sampling instruments.

• Reference to any established methods that were used or research/data collection practices that were followed.

• Explanation of whether the data is georeferenced and, if so, how.

• Explanation of whether the data is disaggregated and, if so, by what groups of characteristics.

• Sample size (e.g., number of the items sampled), sample universe (e.g., the total number of items in the population), and sampling scheme.

Data collection efforts must comply with relevant laws, privacy regulations, and ethical standards. This includes considerations around human subjects, data privacy, and obtaining necessary permissions.

By pursuing one or more of these approaches, the assessment team can fill data gaps effectively while ensuring transparency, accuracy, and compliance. Each approach should be carefully considered based on the available resources, expertise, and the nature of the data gaps identified.

Revisit prior steps

If data gaps are identified during the limitations analysis process, the assessment team may need to revisit prior steps to align the assessment scope with the availability of suitable data. This can involve two main areas of focus: metric selection and landscape boundary delineation.

Below are the steps to follow when considering adjustments in these areas:

• Revisit metric selection: If certain essential metrics lack suitable datasets and there is no feasible way to fill the data gaps, the assessment team may decide to adjust the selection of metrics. The assessment team may devise alternative metrics that can use the available data while still meeting the assessment's objectives. Any newly proposed or adjusted metrics must adhere to the guidelines established for selecting metrics and be submitted for validation.
• Revisit boundary delineation: If multiple essential metrics only have suitable datasets for a specific portion of the landscape, the assessment team may consider adjusting the landscape boundary. The team might choose to focus the assessment on the area that has data available for most or all metrics. Although modifying the landscape boundary is a straightforward process in the LandScale platform, it can have significant implications. It could affect stakeholder engagement, the validity of prior work, and the overall scope of the assessment. These trade-offs should be carefully considered before making such a change. Any proposed changes to the landscape boundary must follow the guidelines outlined for boundary delineation and be submitted for validation.

Example: Revisiting metric selection to better match available data

During an early LandScale assessment in Ghana led by the Nature Conservation Research Centre (NCRC), the assessment team identified the following metrics for the indicator on access to basic services (in LandScale version 0.1):

• School attendance rate (percentage of children).

• Percentage of rural population with electricity access.

• Percentage of rural population with safe drinking water access.

While conducting the data search, the team encountered challenges. They were unable to find suitable datasets for the second and third metrics (percentage of rural population with electricity access and percentage of rural population with safe drinking water access).

Nonetheless, they discovered a research paper about the landscape that provided data on the percentage of households with access to basic needs. The team recognized that this information could be used to assess access to basic services, even though it was not an exact match to the initially selected metrics.

To fill the data gap, the team decided to revise the metrics for the indicator and chose an alternative metric based on the available data. The new metric combined the access to water, sanitation, and electricity into a single measure, which provided a more comprehensive view of access to basic services in the landscape.

This revision allowed the assessment to move forward with relevant data, ensuring the indicator was still meaningful despite the gaps in the original metrics. The team documented the change and submitted the new metric for validation, ensuring that it adhered to LandScale's guidelines.