Review and submit for the local review

The local review is the second step in the Step C validation process. Conducted by designated local experts with landscape knowledge and relevant subject matter expertise, this review evaluates potential data quality limitations flagged during the first LandScale review. It also provides feedback on draft assessment results, strengthening the overall robustness of the findings.

[Insert screenshot of submitting for local review]

Platform support for the local review

The LandScale platform is designed to streamline and support the local review process by enabling reviewers to:

• Register on the platform and log in with their own email address.

• Sign the Local Reviewer Agreement, and review and complete the Conflict of Interest Declaration for Local Reviewers.

• Watch a brief orientation video outlining the local review process and how to use the platform.

• Review the assessment results and documentation for all indicators and metrics within their goal(s), with optional access to results from other goals for additional context.

• Provide structured feedback on the assessment by completing the review questions for each metric.

If any reviewer is unable to use the LandScale platform due to technological, language, or cultural barriers, feedback may be collected through alternative means. Detailed guidelines on accommodating such situations is provided below.

Scope and process of the local review

The local review encompasses all metrics the assessment team seeks to validate for publication at the given time. If the assessment team is pursuing pillar-level validation for Steps B and C, documentation may be submitted only for the specific pillar(s) undergoing validation. Results cannot be excluded from the local review unless the assessment team opts not to publish them, designates them as 'data deficient', or if they have been rejected by the LandScale team during the first review, thereby excluding them from Step C validation.

The assessment team is responsible for coordinating the local review process, which entails the following activities:

1. Select reviewers.

2. Design the review process and format.

3. Orient reviewers.

4. Conduct interactive sessions with reviewers.

5. Collect feedback via standardized review questions.

After these activities are completed, the assessment team must address the feedback provided during the local review. They should also share the final assessment results with local reviewers, accompanied by an explanation of how their feedback was addressed.

Select local reviewers

The assessment team is responsible for identifying and recruiting eligible local reviewers who meet the guidelines outlined in the identify and register local reviewers section. These reviewers should collectively provide both landscape-level knowledge and subject matter expertise across all topics within the assessment scope. To validate their qualifications, documentation such as CVs, resumes, course certifications, or other suitable evidence showing their expertise must be uploaded to the platform. While the selection of at least one reviewer per indicator is required, LandScale recommends engaging a minimum of two reviewers per indicator to ensure a more robust review process.

The selection process begins with the assessment team contacting potential local reviewers to gauge their interest and availability. However, local reviewers cannot be confirmed until the proposed candidates have been reviewed and approved by the LandScale team.

Once a complete set of candidate local reviewers has been identified, it must be submitted to the LandScale team for validation before the local review begins. This submission typically takes place during the first LandScale review for Step C but may occur earlier if desired. If there is uncertainty about whether a candidate meets the selection criteria, the assessment team is encouraged to consult the LandScale team for guidance prior to finalizing the list of candidates.

Design the review process and format

Given the diverse contexts in which LandScale assessments are conducted, our guidelines provides flexibility in designing the local review process to ensure it aligns with the intended purpose and maximises the value of this activity. When designing the local review process, the assessment team should plan how to facilitate each of the following components:

• Orient reviewers to the assessment results: Local reviewers should be introduced to the assessment results and the context in which they were developed. Further details on this component are provided in the orient reviewers section.
• Engage reviewers in discussions about the assessment results: The assessment team may choose to facilitate interactive conversations with local reviewers to explore the quality and interpretability of the results. While optional, this component is highly recommended. Further details are provided in the conduct interactive sessions with reviewers section.
• Complete review questions for each metric: Local reviewers must provide feedback by completing the review questions for each metric within their assigned indicator(s). The default process involves reviewers accessing the LandScale platform, but if this is not feasible, the assessment team should identify an alternative mechanism. Further details on this component are provided in the collect feedback via standardized review questions section.

Once plans for each component have been developed, the assessment team should document the full local review process, including the schedule, logistics, and deadlines. This documentation should specify key dates, such as live orientation or interactive sessions, as well as the final deadline for submitting feedback. Any logistical requirements for live or in-person sessions should also be addressed. These details must be communicated clearly to all local reviewers to ensure alignment and smooth participation.

Orient reviewers

Once candidate local reviewers have been approved by the LandScale team, affirmed their roles and expectations, and registered on the LandScale platform, they must complete a two-part orientation. The orientation entails:

1. General local reviewer orientation: Local reviewers must watch an orientation video provided by LandScale, accessible on the platform. This video introduces reviewers to LandScale assessments, explains the purpose and scope of the local review, and outlines the roles and expectations for local reviewers. It also includes a brief demonstration of the platform’s relevant functions.

2. Assessment-specific orientation: The assessment team is responsible for providing an orientation tailored to the specific LandScale assessment and results being reviewed. This orientation should cover:

   1. An overview of the landscape boundary, assessment scope (indicators and metrics), and the rationale behind their selection.

   2. An explanation of how data sources were selected, along with any challenges or limitations encountered in accessing suitable data for each metric.

   3. A summary of how data were processed and the methods used to generate results.

The assessment-specific orientation can be delivered live, via a pre-recorded video, or in another suitable format. The assessment team may choose to provide a single orientation session to all local reviewers, covering the entire assessment, or tailor sessions for each goal or pillar to give reviewers more in-depth insights into their specific topics while avoiding information overload. If interactive sessions form part of the local review process, the orientation to the draft assessment results may be incorporated at the start of these sessions.

While the orientation serves as an important foundation for local reviewers, it must remain factual and objective, summarizing the assessment process and results as documented by the assessment team. The assessment team should take care to avoid introducing bias or influencing reviewers' perceptions of the draft results or their findings.

Conduct interactive sessions with reviewers

Interactive sessions are strongly recommended, though not required, to help local reviewers thoroughly understand and assess the draft results. These sessions also allow reviewers to triangulate their observations and feedback with those of other reviewers. The sessions may follow one or both of the formats below:

• Bilateral: The assessment team meets with each local reviewer individually to gather their preliminary feedback, answer questions, and provide further context on the assessment process. In this case, the suggested format is one-on-one remote meetings.

• Dialogue: The assessment team facilitates discussions among multiple local reviewers, enabling them to exchange views and, where appropriate, reach consensus on various aspects of the review. In this case, the suggested format is either in-person or virtual meetings, utilizing facilitation techniques and live input tools available in virtual meeting platforms to foster meaningful interactions.

The purpose of these sessions is to inform the individual feedback that each reviewer will provide in response to the review questions. By participating in interactive sessions, local reviewers can gain a deeper understanding of the draft results, their level of accuracy, and any potential limitations. However, these sessions do not replace the formal mechanism for individual feedback.

The assessment team is encouraged to document any interactive sessions—such as through meeting or workshop minutes—as part of the overall assessment documentation.

Collect feedback via standardized review questions

Local reviewers will provide feedback by answering a set of review questions for each metric within their review scope. These questions will be accessible once reviewers have registered on the platform and completed the Conflict of Interest Declaration. The assessment team should inform reviewers when they should complete the questions, based on the overall review process.

The review for each metric includes the following standard questions:

1. To the best of your knowledge, do the results reflect the actual conditions in the landscape?

2. Do you have any concerns about the result, underlying data quality, or limitations documented?

[Insert screenshot of review questions for local reviewers]

In limited circumstances, additional questions may be added that are unique to particular metrics. Specifically, if the first review of Step C by the LandScale team identified potential concerns or limitations with the quality of results or underlying data sources that were not resolved by the time of the local review, the LandScale team may include questions for local reviewers on these potential concerns or limitations. Reviewers should answer these additional questions for any metric for which they are included.

Once a local reviewer has completed the review questions for a given metric, they will have the option to either approve or reject the metric. If rejected, this indicates that the local reviewer believes the metric should not proceed. However, even in the case of rejection, the metric can still be submitted for the second LandScale review, during which the LandScale team will assess whether the feedback from the local reviewer has been adequately addressed.

Whenever possible, local reviewers should respond to the review questions directly on the LandScale platform. If this is not feasible for certain reviewers, a member of the assessment team may receive the reviewers' responses through conversation, create a local reviewer account, and serve as a scribe to enter the responses into the platform on behalf of the reviewer. In this case, the scribe must directly transcribe and enter the reviewer’s responses without alteration. Prior to submitting the responses, the scribe should read them back to the reviewer to confirm that they have been entered correctly. Additionally, the username for this local review account must include the names of both the reviewer and the scribe, for example, '[name of the scribe] on behalf of [name of the local reviewer].'

[Insert screenshot of scribe account]