United Nations Economic Commission for Africa

Evaluation Function

The M&E functions in UNECA are located within the Office of Strategic Planning and Programme Management (OPM). OPM ensures that minimum requirements are set by developing and updating UNECA'™s specific RBM/M&E policy, guidelines, frameworks and tools consistent with UN Secretariat-wide ME&E standards and norms, including ensuring that gender equality and youth concerns are reflected and mainstreamed in UNECA'™s M&E activities. OPM is also mandated to ensure oversight of UNECA interventions by maintaining the adequacy, accuracy and credibility of the M&E system.

The Director of OPM is directly accountable to the Executive Secretary of UNECA. In consultation with the Director of OPM, the Chief of the evaluation function has the authority to sign off on and distribute evaluation reports to the governing body and/or executive head. The Chief has limited  control over evaluation expenditure which is allocated by the Budget Committee. Evaluators sign a statement of potential conflict of interest, yet there are no clear rules and mechanisms in place that allow evaluators to report discreetly on cases wrongdoing.

Promoting a Culture of Evaluation in UNECA

On-the-job training is available for  entities within the Commission, which include formal seminars and workshops and coaching and/or peer groups for learning about results-based management and the importance of evaluation. As of 2012, OPM will intensify its provision of training workshops to all levels of the organization, including directors, middle-managers and programme managers and associates. 

Snapshot

Institutional Set-Up

• The evaluation function is combined with the monitoring function and located within the Office of Strategic Planning and Programme Management (OPM)

Staff as of 31 December 2011

• Unit Head: Male (Chief, Programme Planning and Monitoring and Evaluation Section)
• Evaluators: 1 male
• Support staff: 1 female and 1 male
• Decentralized evaluation staff: none

Evaluations conducted or commissioned by the unit in 2011

• 12 evaluations; seven of these were conducted by external consultants and five evaluations were managed or led by the central evaluation office (three of which were classified as corporate/thematic/strategic evaluations, and two of which were operations evaluations).

Evaluation expenditure in 2011

• US$122,600

Priorities

• Carry out evaluations of ECA's flagship publications and develop detailed evaluation guidelines (handbook) in line with UNEG's norms and standards
• Strengthen the culture of results and evaluations at ECA (through, trainings, workshops and seminars) 

Independence

ECA is determined to make planning for results and M&E the driving force behind the Commission’s results culture. To this end, the Commission has been operating under an explicit policy statement, the "€œUnited Nations Economic Commission for Africa Results-Based Management Policy Note"€ which has been in effect since 2008. The Policy was updated in 2011 in light of the United Nations Norms and Standards and re-issued in February 2012.

Evaluation at UNECA takes place at programme, sub-programme and project levels. Within the UNECA evaluation function, two categories of evaluation are distinguished: External evaluations, which are managed and conducted by entities outside ECA such as OIOS or JIU to demonstrate accountability to donors, member States or other external stakeholders; and internal evaluations, which are managed by ECA and may be conducted by its staff and/or external consultants.

The core tasks of the ECA evaluation function include: (i) preparing annual evaluation plans; (ii) drafting and reviewing terms of reference and evaluation designs; (iii) carrying out evaluations and self-assessments; (iv) reviewing evaluation reports drafted by consultants; (v) providing guidance to consultants; and (vi) following up and reporting on the implementation of recommendations. Other tasks include: RBM; monitoring; training; developing evaluation capacities (in-ECA); quality assurance processes; policy/strategy development; and programme/budget development. 

Agenda Setting & Evaluation Planning

At the beginning of each biennium OPM is mandated to issue an evaluation plan after a consultation with sub-programmes. The work programme covers timeliness of evaluations with a view to inform decision-making (partially), priority areas most in need of evaluation during the work programme cycle, and specifies necessary resources for evaluation. The work programme is to be derived from the Approved Programme Budget and consists of mandatory and discretionary internal evaluations: Mandatory internal evaluation is a self-assessment carried out by UNECA at the request of statutory bodies such as the Conference of Ministers, OIOS or any relevant body overseeing the activities of UNECA, while discretionary internal evaluation is a self-evaluation undertaken by UNECA or its sub-programmes primarily for the organization’s own use as an integral part of the management process.The work programme is submitted for review and/or approval by the Head of the Commission.

Stakeholder Involvement and Promoting National Evaluation Capacity

Stakeholders are systematically consulted in the conduct of evaluations; however the Fact Sheet reports that stakeholders are not consulted in the planning/design of or follow-up to evaluations. Peer reviews or reference groups composed of external experts are used in the evaluation process and evaluation teams include professionals from the countries or regions concerned.

Quality Assurance

The standards that guide evaluation in ECA include accuracy standards, feasibility standards, utility standards, and proprietary standards. Evaluation Terms of Reference, and Inception Reports where applicable, are systematically checked against the UNEG Checklists. In certain cases, expert panels are asked to provide additional quality assurance. 

Use of Evaluation

An explicit response (partly) to the evaluation is issued by the Head of the Commission, the Head of Operations and the Head of the entity that was evaluated, in the form of a Management Response Matrix, a section in an updated project/programme design document and an Action Plan. It is reported to Oversight bodies. There is systematic follow-up by management and the evaluation entities of the implementation of the recommendations.

The UNEG Self-assessment indicates that evaluation results are partially disseminated within the Commission, but are not disseminated outside the organization. The Policy Note indicates that findings and lessons should be accessible to target audiences in a user-friendly way and reports should be subject to a dynamic dissemination strategy tailored to the audience of that specific evaluation report. The Policy also notes that OPM has the mandate to ensure the highest standard in accessibility and presentation for published reports, providing additional learning products based on evaluations, using a range of channels to reach target audiences.  As reported in the UNEG Self-assessment, UNECA is currently systematically extracting lessons (partly) from evaluations and communicating them through information briefs, abstracts, press releases, meetings with senior management and operations management, and annual evaluation reports. Evaluations are not available in an online repository.

Joint Evaluation