The results reporting framework
In the 2009-2014 funding period, both the beneficiary and donor countries are committed to managing for results. A 'results framework'- setting out defined performance indicators, baseline data and targets - is in place. This serves as a platform for tracking achievements and results.
The Regulation for the Grants - which outlines 10 priority sectors and 32 programme areas - is the framework for the management of the contributions from the EEA and Norway Grants.
Reporting framework organised into four levels
Under the EEA and Norway Grants we have identified a set of objectives, outcomes and outputs (with corresponding indicators) which we will seek to influence. We track indicators at four reporting levels. Achievements are compared against baseline data and targets at the national, sector and programme levels and we use these to assess our operational efficiency. Over time these indicators highlight trends, successes and bottlenecks which require further attention. By reporting against indicators,the reporting is evidence based.
- Level 1:What development progress are the beneficiary countries making regarding the overall objectives of the Grants?
- Level 2: What are the Grants’ results in the beneficiary countries? (effectiveness)
- Level 3: What is the strategic focus of the Grants?
- Level 4: What is the operational efficiency of the Grants?
Development of the reporting framework has been informed by best practice of other international grant management institutions and the OECD as well as by recommendations in reviews and evaluations of the previous funding period.
Level 1: What development progress are the beneficiary countries making towards reduced economic and social disparities and strengthened bilateral relations?
Level 1 of the framework tracks key developments in the beneficiary countries through selected development indicators at national level and in sectors of relevance to the Grants. In addition, trends in bilateral relations are monitored.
Monitoring of national development trends gives us an opportunity to understand the synergy between the EEA and Norway Grants and other funding schemes, and the connections between the two overall objectives and the sectors and programmes supported by the Grants. The national development trends also provide us with a justification for the Grants; WHY we provide the Grants.
The development results cannot be attributed to the Grants as these are small in the larger picture. No evidence will suggest that results achieved at the national and sector levels are the direct contribution or attribution of the Grants. We will nevertheless monitor these developments as they provide the context for the Grants, and allow us to assess the relevance of the interventions. We may, in some cases, be able to assume that the Grants indirectly contribute to national improvement. Data will be drawn from Eurostat and other international and national statistics.
Macro indicators at national and priority sector levels as well as bilateral data are tracked and reported to indicate the context and relevance of the Grants.
Indicators that show:
Regulation Article 2.2
Sources of verification:
Reports by the FMO:
Level 2: What are the Grants’ contributions in the beneficiary countries?
At Level 2, we measure the development effectiveness of the Grants. We track the programmes’ progress towards theiroutcomes and outputs. We want to know WHAT the results of the Grants are.
Programme and project results chain
Higher-order, long-term objective to which a programme is expected to contribute and the effects on identifiable population groups. These effects can be economic, socio-cultural, institutional, environmental, technological or other types, and have a relationship with national priorities and the objectives of the Grants.
The likely short-term and medium-term effects of a programme responding to national sector priorities and local needs and the outcomes of the Grants.
Deliverables (products, goods, services) which result from an intervention in a programme.
Necessary actions to achieve outputs.
Funds, personnel, expertise, knowledge.
Each of the 32 programmes areas has one objective and a certain number of outcomes (in all 160 programme outcomes + bilateral relations outcomes). In addition, each programme has defined at least two outputs per outcome.
Each outcome and output requires one or several indicators. The Programme Operators, the national Focal Points and the Financial Mechanism Office (FMO) report on the programme outcomes and outputs, by tracking the indicators that are concluded in the programme agreement and entered into a common Documentation, Reporting and Information System. Where possible, the indicators will be gender-disaggregated.
The FMO will track, analyse and report aggregate programme results.
Annual risk analyses provide indications of the likelihood of programmes reaching their outcomes and outputs.
What are the results of the Grants in the beneficiary countries towards programme outcomes and outputs, including bilateral outcomes and outputs?
Regulation Article 2.3.2
Sources of verification:
The national Focal Point’s annual Strategic Report.
The Programme Operator’s Annual Report.
The Donor Programme Partners’ reports.
Monitoring, audits, reviews and evaluations.
Documentation, Reporting and Information System DoRIS.
Reports by the FMO:
Annual Report, fact sheets on sectors and countries, results reports to the donors and the public.
Level 3: What is the strategic focus of the Grants?
At Level 3, data on the strategic focus in the portfolio are available to monitor progress in the allocation and spending funds in the various areas. We want to know in WHICH way the funds are spent.
Our statistical data base will provide us with data on the composition of the portfolio regarding distribution of funds across countries, priority sectors, programme areas, geographical areas, policy markers, sector codes, size of projects and programme and types of implementing agencies and donor partner institutions.
We may for instance assess the extent to which civil society is associated with the Grants or the volume and numbers of bilateral relations. By monitoring gender dis-aggregated indicators, we are in a position to report on the Grants’ contributions to gender equality by sector, programme area and country – at Level 3 we use input data, mainly distribution of funds, as a basis for our analysis.
The FMO statistical manual shows the kind of data we will be able to collect through the Documentation, Reporting and Information System. Data on the strategic focus of the Grants allows us to monitor whether the portfolio is in accordance with the planned priorities, to redirect efforts as and when needed and to inform about the portfolio.
What is the strategic focus of the funds?
Distribution of funds and other data that will be tracked at programme and project levels:
Variables only at programme level:
Variables only at project level:
Regulation Article 2.
Sources of verification:
Annual Strategic Report from the National Focal Points.
Documentation, Reporting and Information System (DoRIS) – project and programme data.
Reports by the Financial Mechanism Office:
Annual Status Report.
Regular reports to the donors and public.
Level 4: What is the operational efficiency of the Grants?
At Level 4 we want to know if the operations are managed effectively and efficiently. HOW do we work?
Reports provide data on financial forecast and payments, spending patterns, approval and implementation rate, other progress information, possible irregularities, due dates and completion rates. Monitoring and evaluations will show us if we are efficient, flexible and able to change.
By regularly looking at the operational performance, we can highlight progress, bottlenecks, and implementation shortcoming requiring managerial attention and follow-up.
What is the operational efficiency of the Grants?
Provision of data and analyses on progress, finance and administration and management.
Types of variables that will be tracked:
Assignment Letter no 1/2011 from the Norwegian Ministry of Foreign Affairs to the FMO.
Sources of verification
Mainly the Documentation, Reporting and Information System.
External reviews and evaluation.
Reports by the FMO:
Annual Status Report.
Regular reports to the donors.
The commitment to managing for results requires a systematic application of statistics, and the need for statistics is brought into the mainstream of the present planning phase of the Grants 2009-2014. The FMO has developed a statistical manual for the Grants 2009 – 2014.
Financial data are not included in the manual. The manual is primarily a tool for the Programme Operators, but will also be useful for the national Focal Points, the Donor Programme Partners, the FMO and the donors. The Programme Operators will enter data and report on projects and programmes, the national Focal Points will report on programmes at a more aggregate level and national trends regarding social and economic disparities and bilateral relations, whilst the FMO will monitor and report data on projects, programmes, country and priority sector trends, as well as the overall Grants.
Indicators and sector codes
The FMO has developed a list of standard, quantitative indicators towards the 160 programme outcomes the objective of bilateral relations. This list will be revised. The list is included in the Documentation, Reporting and Information System. In addition the Programme Operators may define their own quantitative or qualitative indicators.
Core indicators for publicity purposes
It will be possible to track all indicators in the the Documentation, Reporting and Information System.However, in a single “corporate” framework it is not meaningful to track all the individual indicators (several hundred) on a regular basis. Instead the FMO will select a range of core indicators through an analysis of expected results outlined in the programmes. These will serve as a basis for public communication. We will try to choose a small number that can immediately be understood by the public. These indicators will be summed across programmes and countries to prove evidence of what the Grants achieve and contribute to.
The sector codes are developed especially for the Grants. The system is based on OECD’s system for sector codes. Each project is given a sector code by the Programme Operator.
National and priority sector Macro indicators (not in the Documentation, Reporting and Information System)
To analyse country trends regarding social and economic disparities and bilateral relations, as well as priority sectors, the FMO has set up a small data base with data from 2005 – 2008 – 2010/11 (latest available). Whether or not this will be entered in the Documentation, Reporting and Information System will be assessed later. These data are used in the Strategic Report by the National Focal Points and in the Annual Report from the donors to their parliaments and the public.