Interim Evaluation of the Priority Area “Administrative Capacity”

Objective

The objective of the interim evaluation was to assess the achievement of the first two objectives of the administrative capacity priority area, as well as the implementation and effectiveness of the activities funded to achieve these objectives, while also considering other activities with similar goals.

Methodology

The evaluation applied an innovative evaluation model in which the activities under the administrative capacity priority area were grouped into three areas: development capacity of the government sector, leadership capacity of the government sector, and the quality of policymaking.
The evaluation model takes into account the key prerequisites of good governance, which emerge through the interaction of these three main components.

Main results

  • Measures implemented to strengthen the development capacity of the government sector have produced the expected results and remain relevant. However, a certain fragmentation and duplication in strategic planning has been identified as a problem, limiting the potential synergy between different measures.
  • Leadership capacity within the government sector has improved as a result of the supported activities, and implementation is progressing according to plan.
  • The quality of policymaking has also improved due to the supported activities: cooperation capacity has increased, inter-institutional collaboration has strengthened, and the substantive quality of impact assessments has improved.

Challenges

Currently, there is no long-term national development plan for strengthening administrative capacity, which means that the continuous development of administrative capacity is not guaranteed. It is recommended to develop a long-term, cross-sectoral development plan within the framework of the Estonia 2035 strategy to support the development of administrative capacity for both the central government and local governments.

Evaluation Methods: The evaluation included an analysis of financial, output, and outcome indicators of the measures and their activities. Focus group discussions and individual interviews were conducted. In addition, information was collected and analyzed through document analysis.

Project clients: Government Office of Estonia and the Ministry of Finance
Project implementers: LevelLab (lead partner), Institute of Baltic Studies, and Technopolis Group Estonia
Study duration: 03/2019 – 12/2019

Interim Evaluation of the Implementation Plan for EU Cohesion Policy Funds

Objective

The purpose of the interim evaluation was to assess the relevance, effectiveness, impact, and sustainability of the implementation plan and the activities carried out within it. The evaluation covered all 14 priority areas of the implementation plan.

Results

The interim evaluation concluded that the sectoral objectives and activities of the implementation plan remain relevant today, and there is no need for significant changes to achieve the objectives of the implementation plan.

As of December 31, 2018, 32% of the €4.6 billion from the EU Cohesion Policy implementation plan for 2014–2020 has been disbursed, totaling €1.5 billion. The support received from the EU Structural and Investment Funds has clearly contributed to achieving the objectives of all priority areas: the funding has a direct impact on increasing employment, improving public health, developing the living environment, and achieving other objectives. The impact of Structural Funds is long-term and manifests in changes at various levels. The operational program has had a significant impact on achieving Estonia’s key strategic objectives.

Challenges and Recommendations

Regional development balance remains a significant challenge. Overall, regional disparities remain significant, and there has been no noticeable shift towards more balanced territorial development. Close to 50% of EU funds have been invested in Harju and Tartu counties, leading to a widening of regional disparities. EU funds should be directed towards preventing problems and developing comprehensive solutions in regions, with an increased focus on the role, awareness, and competence of local governments in regional investments.

Additional funds from the performance reserve should be directed towards preventive measures and value-added activities such as research and development, innovation in entrepreneurship, education, and regional development.

Research methodology: focus group interviews, individual interviews, workshops, document analysis (documentation of projects, measures, and implementation plans), and quantitative analysis (financial, output, and outcome indicators of measures).

Project contracting authority: Ministry of Finance
Project implementers: Technopolis Group Estonia (lead partner), Ernst & Young Baltic, Praxis, Baltic Institute of Research and LevelLab
Duration: 11/2018 ‍– 04/2019

View the final report

Business Analysis of AS GoTravel

Background

AS GoTravel is an Estonian-owned travel agency that has been operating since 1992. The agency has offices in six Estonian cities: Tallinn, Tartu, Pärnu, Viljandi, Rakvere, and Jõgeva. The company serves both private and business clients and offers both domestic and international tourism services.
The company’s information system has been developed based on operational needs and investment capacity. The greatest focus has been placed on developing the public web environment as the main sales channel for package tours.

Objective

The objective of the business analysis was to describe the current state of the company’s information system and its connections with the company’s business processes, and on this basis provide development proposals for improving the information system and adapting the related action plan.

Benefits

As a result of the business analysis, GoTravel received development proposals for improving its information systems in order to simplify customer communication, internal information sharing, the signing of various contracts, and the use of the loyalty program, as well as to enhance the security of the public website.
The analysis also resulted in the development and implementation of a payment link solution for PDF invoices, created in cooperation between LevelLab and OÜ WeIT, which significantly simplifies and speeds up invoice payments for customers.

Project client: AS GoTravel
Project implementer: LevelLab
Study duration: 09/2019–11/2019

Overview of Estonia’s Research and Innovation System

Objective

The purpose of the background report was to provide a comprehensive overview and assessment of Estonia’s research and innovation system to the European Commission. The overview included the structure and organization, management and funding, development, and other aspects of the research and innovation system. An assessment was provided on the management of the Estonian research and innovation system, the funding system for research and higher education, and external financing over a period of 10 years.

Main results

  • The most significant driver of economic development and growth is innovation, particularly innovation based on research and development (R&D).
  • The capabilities necessary for creating innovations are needed for the national innovation system to learn, develop the necessary human capital, stay abreast of international advances in science and technology, and maintain its productivity and competitiveness.
  • Innovators typically do not innovate alone but within national and international innovation systems that involve multiple participants and institutions. Therefore, policymakers must ensure that the combination of policies related to various parts of the system is balanced and that interventions across the entire innovation system are coordinated.

Methodology

Quantitative data and analysis, as well as document analysis, were used in the preparation of the background report.

Project contracting authority: European Commission
Project implementers: Technopolis Group Estonia and LevelLab
Duration: 10/2018 ‍– 02/2019

View the final report

Interim Evaluation of the Ministry of Education and Research R&D Programmes “Estonian Language Technology 2018–2027” and “Estonian Language and Culture in the Digital Age 2019–2027”

Background

The interim evaluation focuses on the Ministry of Education and Research programmes “Estonian Language Technology 2018–2027” (EKT) and “Estonian Language and Culture in the Digital Age 2019–2027” (EKKD), which support research and development related to the Estonian language and culture in the digital era. The programmes encompass both language technology development and research on language and culture, and their implementation is shaped by changes in the research and technological environment, including the rapid development of artificial intelligence and large language models.

Objective

The purpose of the study is to assess the effectiveness of the EKT and EKKD programmes to date by mapping the current state of the field and evaluating the extent to which the programmes’ objectives have been achieved, as well as their broader contribution to research and development in the areas of the Estonian language, culture, and language technology. In addition, the study aims to provide recommendations on how the Ministry of Education and Research should further develop research and development in the language field and whether and how it would be appropriate to fund such activities after the programmes end in 2027, taking into account global trends, limited financial resources, and the need to avoid duplication with other measures.

Methodolgy

The evaluation is based on a multi-layered data collection strategy that combines document analysis, interviews, focus groups, and the analysis of quantitative indicators.

Benefits

The interim evaluation provides a comprehensive overview of the programmes’ implementation to date and their contribution to achieving the set objectives. The evaluation report includes an assessment of programme performance as well as recommendations and proposals for future funding and operational models following the conclusion of the current programmes.

Project Client: Estonian Research Council (ETAG)
Project Implementer: LevelLab
Study Duration: 07/2025–02/2026

Model for Evaluating the Impact of Cultural and Sporting Events

Background

Cultural and sports events have diverse impacts, including economic, social, cultural, and environmental aspects. These events increase Estonia’s attractiveness in the eyes of both domestic and international tourists, promote local culture and sports life, and help enhance Estonia’s reputation worldwide. Various models have been developed to assess the socio-economic impact of events, but they often lack a social dimension. Additionally, the impact of media visibility associated with events on the image of tourist destinations is inadequately evaluated, and the application of these methodologies for events held in larger cities is complex.

Objective

The aim of the study is to develop a methodology (tool) for assessing the socio-economic impact, including the influence on Estonia as a tourist destination, of internationally appealing cultural and sports events held in Estonia. This includes assessing the image of Estonia as a tourist destination, and allowing for future reassessments. In addition, the methodology will be piloted based on 20 events, and proposals for the implementation of the methodology will be developed.

Methodology

LevelLab will pilot the model at 20 events, where three target groups will be surveyed: foreign and domestic tourists, the local community in the areas where the events take place, and businesses. In addition, personal interviews will be conducted with organizations organizing the events and representatives of local governments in the event locations, focus group interviews will be held, and secondary quantitative data will be analyzed.

Benefits

A decision-making tool  for cultural management will be developed to facilitate the planning of individual events. In addition, the tool can be used to enhance Estonia’s positioning and reputation as a tourist destination.

Project ordering institution: Ministry of Economic Affairs and Communications
Project implementers: LevelLab and Roheline Rada
Duration: 08/2023-12/2024

The processing tool for admission statistics at the Estonian Academy of Security Sciences

Background

The Estonian Academy of Security Sciences wanted to implement an automated tool for processing admission statistics queries in the SAIS information system, which simplifies the processing of admission statistics, presenting them as compact data tables and visualizations.

Objective

The aim of the proposed solution was to create an Excel workbook, which takes as input structured data tables (for example, .csv extracts from the SAIS admission information system) or PowerQuery data queries from SAIS.

Methodology

The structure and functionality of the solution were designed during the requirements gathering process, which involved document analysis and interviews with representatives from the Estonian Academy of Security Sciences. During the interviews, the structure of the output worksheets, the formatting of tables, charts, and diagrams, as well as other necessary information, were clarified.

Benefits

The tool helps reduce the time spent on compiling and analyzing admission statistics at the Estonian Academy of Security Sciences, thereby ensuring a faster and more efficient process.

Project contracting institution: Estonian Academy of Security Sciences
Project implementer: LevelLab
Duration: 10/2023 ‍– 12/2023

Analysis of the Implementation of the Qualifications System

Background

The qualifications system is part of the Estonian qualifications system – it connects the education system with the labor market and contributes to the development, assessment, recognition, and comparison of work-related competencies. The implementation of the qualifications system has been regulated by the Vocational Qualifications Act since 2001, and the system consists of four parts: the qualifications framework, vocational standards, awarding of vocational qualifications, and the vocational register.

Objective

The aim of the study was to determine the expectations of the participants in the qualifications system, their readiness for implementing the reform, and the transition to a skills-based approach, as well as to identify the potential challenges and risks associated with the planned innovations.

Methodology

During the study, the qualifications system used in Estonia was compared with the experience of other countries. For assessing the readiness and expectations of stakeholders in the qualification system, interviews and focus groups were conducted with direct and related stakeholders of the qualification system: applicants for certification, certifying bodies, educational institutions, learners, stakeholders involved in developing occupational standards (including employers and professional associations), career advisors, qualification system specialists, and policymakers. During the study, risk assessment was also conducted to ensure the realism and feasibility of the innovations.

Benefits

As a result of the study, input was obtained for the renewal of the legal system underlying the implementation of the qualifications system and for the implementation of a skills-based qualifications system.

Project contracting institution: Ministry of Education and Research
Project implementer: LevelLab
Duration: 03/2023 – 10/2023

View the final report