NATIONAL FRAMEWORK FOR THE ASSESSMENT OF STRUCTURAL INSTRUMENTS 2014-2020

Bibliografie

The national framework for Evaluation of Structural instruments is aligned with the European legislation in force but also with the public policy framework made by Romania, such as in other Member States, concerning the allocation and implementation of funds. 
Thus, the evaluation policy of ESIF in Romania implements Articles 54 to 57 of Regulation EU No 1303/2013, which sets out the general framework for the evaluations of the 2014-2020 programming period. 
Article 54 of Regulation No 1303/2013 establishes the objectives of the evaluation: 
1.    To improve the quality of the design and implementation of programmes; 
2.    To assess the effectiveness, efficiency and impact. 
The ex ante evaluations, carried out for all OPs/national plans for 2014-2020 are associated primarily with the first objective (Art. 55 (1) of Regulation (EU No 1303/2013), while the evaluations during the implementation period and ex-post relate, in particular, to the second objective (Arts 56 (3) and 57 (1)). 

Art. 54, Art. 56 (3), Art. 57 (1) of Regulation No. 1303/2013 provide further information on how the impact of programmes, in particular by taking into account: 

  • The Europe 2020 strategy’s targets, emphasizing the objectives;
  • GDP and unemployment in the coverage of the Programme, taking into account the size of the project;
  • The objectives of each priority.

Romania, as a Member State, is responsible for the ex-ante evaluation and the evaluations during the period of implementation of the programmes (including impact assessments). The ex-post evaluation will be undertaken by the European Commission but also possibly at national level, in close cooperation with the Commission. From the perspective of the public policy framework, during the successive programming periods, including 2014-2020, funds were allocated and implemented on three levels, consisting of: 

  • Macro level, encompasing the whole country: country strategies, such as the Partnership Agreement for 2014-2020 or the National Strategic Reference Framework 2007-2013;
  • Meso level which operationalises the strategy’s priorities as well as operational programmes which establish specific objectives and actions;
  • Micro level, represent the projects contributing to the objectives of the operational programme.

Micro level — projects funded
For the 2014-2020 programming period, it is proposed a systematic approach to assessment at micro level in the evaluation plans of the operational programmes. In fact, this micro level is fully integrated in the meso evaluation, serving the needs of each priority axis and of each specific objective. As for most programmes, micro assessment of all projects is not feasible, a sampling strategy for the identification of a representative group of projects that must be developed. 

Meso level — operational programmes
Concerning the assessment of the OP level (meso) under Regulation No 1303/2013, each priority shall be reviewed at least once during the implementation of the Programme in contributing to the specific objectives. There are four dimensions to be taken into account for all evaluations: 

  • Spacial
  • Sectoral
  • Target groups
  • Institutional

In terms of evaluation methods at meso level, counterfactual impact evaluations will be used whenever possible to answer questions on the net effects on target groups. However, to analyse the effects on different territories and sectors, a counterfactual approach might prove inappropriate. Thus, service providers will use other approaches, based on their own experience and international practice. 
Evaluations carried out at both micro and meso covers the following seven evaluation questions: 
1.    What is the progress which can be observed in areas, sectors and target groups of specific objectives, since the adoption of the intervention? 
2.    Are there unintended effects, positive or negative? 
3.    To what extent progress observed can be attributed to the intervention? 
4.    These effects go beyond the border area or sector of intended effect or impacts other non-target groups of the intervention? (spill-over effect)
5.    The extent to which the effects are sustainable over a longer period of time.
6.    What mechanisms have facilitated/hindered the effects? What are the key contextual features of these mechanisms? 
7.    Whether and how things could have been done better? 

Macro level — Partnership Agreement

Macro evaluations carried out at the level of the Partnership Agreement concern three global topics: 

  • the wider impact of the policy,
  • development needs and relevance
  • effectiveness of implementation mechanisms.

The effects of the broader policy concerns the extent to which the ESI funds contribute to European and national policy objectives, such as the Europe 2020 strategy, economic, social and territorial cohesion, the thematic objectives and the country-specific recommendations. The main evaluation methods are modelling techniques and Evaluation Summary Reports at macro and meso, according to the legal assessment.  

The assesment regarding the relevance of development needs will look at the changes in development needs as a result of changes in the policies and priorities of European and in the national socio-economic development that have occurred since the adoption of the Partnership Agreement. The assement will draw conclusions on the continued relevance of interventions included in the programme documents. Providers will update the socio-economic analysis of the PA and SWOT analysis. 
Assessing the effectiveness of the implementation mechanisms — The third global topic at macro-level occurs on the basis of two aspects raised during the previous programming period: 

  • Similarity of the implementation arrangements between programmes. Hence, this subject has proved to be systemic, including horizontal elements;
  • Excessive efforts for evaluating the implementation of the OP, requiring sustained coordination as regards the planning and implementation of their recommendations.

The key elements of the evaluation under the third global topics refer to partnership, principles and horizontal policies, performance framework, the capacity of authorities and beneficiaries, administrative burden, complementarity and coordination of funds.