International Maritime Organization

Evaluation Function

The evaluation function in IMO is shared by the Internal Oversight and Ethics Office (IOEO) and the Technical Co-operation Division (TCD). The IMO evaluation policy was published in 1988 but has not been updated in light of the UNEG Norms and Standards for Evaluation in the UN system and is not publically available. Furthermore, the policy has not been fully used since 1998 with the incorporation of the three oversight functions (i.e. internal audit, evaluation and investigation) into one office. Since then, the evaluation function of the office is only focused on the fulfilling the instruction of the Secretary-General on ex-post evaluation of training events.

IOEO is responsible for ex-post evaluation of training events. TCD is responsible for Impact Assessments which are carried out as an external exercise every four years. Impact Assessments are financed through the un-programmed reserve of the TCD Fund which, in 2011, was a cost of USD 126,383.

In 2011, there were three internal oversight and ethics staff members and a central evaluation expenditure of USD 30,260. Three external consultants were engaged by TCD to conduct the 2011 Impact Assessment.  

Snapshot

• Evaluation Policy:
• Not available publically.
• Priorities:
  
• Ex-post evaluation of training events – analysis of replies and reporting of findings.
• Impact assessment every four years of technical co-operation projects.
• Human Resources:
•  Unit Head: M
• Internal Oversight Officer: Total 1 (F)
• Support staff: Total 1 (F)
• Decentralized evaluation external consultants: Total 3; F = 2 and M=1
• Evaluations produced per year by the central unit and by decentralized units
  
• Three operations evaluations at the central level.
• One decentralized Impact Assessment conducted every four years.
• Key resource:
  
• No website available.

Independence

The evaluation function in IMO reports directly to the Office of the Secretary General. The Senior Deputy Director/Head of IOEO has the authority to sign off and distribute evaluation reports to the IMO Secretary-General without prior clearance from other parties, within or outside the organization. 

Agenda Setting & Evaluation Planning

The IMO prepares an annual work programme, which addresses the timing of evaluations with a view to inform decision-making; priority areas most in need of evaluation; and specification of resources for evaluation.

The work programme for ex-post evaluation of training events is submitted to the IMO Secretary-General for review and/or approval, whilst the work programme and terms of reference for impact assessment is submitted to the IMO Governing Body for approval. 

Stakeholder involvement and promoting national evaluation capacity development

Stakeholders are systematically consulted in the planning/design, conduct of and follow-up to evaluations. Peer reviews or reference groups composed of external experts are not used in the evaluation process. Evaluation teams include professionals from the countries or regions concerned.

Quality Assurance

The quality of evaluation reports is systematically controlled by a set of quality rules that cover evidence-based evaluation report, findings and recommendations, conciseness of executive summary, methodology and limitations, coverage of TORs, evidence of results, quality of recommendations, and how the evaluation dealt with differences in opinion.

Use of Evaluation

The reports on the findings and analysis of ex-post evaluation questionnaire responses are submitted to the IMO Secretary-General.

The IMO Secretariat issues an explicit response to the Impact Assessment in the Technical Co-operation Committee document “Secretariat Response to the Consultant’s Report: Impact Assessment Exercise”. This document contains comments on findings and recommendations, and proposed follow-up actions. It is reported to the IMO Council as part of the report of the Technical Co-operation Committee.

Evaluation results are disseminated widely both internally and externally. Lessons are systematically extracted and communicated through meetings with senior management.

A repository of evaluation studies is available which catalogues past IMO evaluation reports that can be requested from IOEO.

Joint Evaluation

IMO are not currently involved with joint evaluations