Evaluation of the Effectiveness of the ProgeTiigri Program Activities

Background

The ProgeTiiger program was launched in 2012 in order to develop students’ digital competence and technological literacy and to understand the nature of technology, its application, and development opportunities. The program’s resources have supported the development of teachers’ digital competence, promoted networks of teachers and instructors in the field, and co-financed the acquisition of equipment necessary for teaching.

Objective

The aim of the study was to assess how the ProgeTiigri program has contributed to the improvement of the quality and accessibility of technology education from 2018 to 2021. In addition, recommendations were provided for modifying program content activities and for follow-up actions.

Methodology

The study utilized both qualitative and quantitative data collection and analysis methods (document analysis, surveys, focus group and individual interviews). The survey involved teachers, educational technologists, and educational institution leaders from Estonian general education schools. Personal interviews were conducted with employees of the Ministry of Education and Research, the Education and Youth Board, and the ProgeTiigri program managers. The focus group was conducted with representatives and experts from universities, as well as with educational leaders and/or teachers actively involved in the program.

Benefits

As a result of the study, feedback was provided on the activities of the ProgeTiigri program during the period 2018-2021, and an assessment was made of how purposeful the activities have been and what improvements have been made in the quality and accessibility of technology education. Additionally, recommendations were provided on what changes could be made to the content activities of the program and how to proceed with the program.

Project contracting institution: Education and Youth Board
Project implementer: LevelLab
Duration: 01/2023 ‍– 05/2023

View the final report

Evaluation of Performance Indicators for the Estonia-Latvia Program

Background

The Estonia-Latvia Cross-Border Cooperation Programme (https://estlat.eu/en) involves the southern and western parts of Estonia and the northern and western parts of Latvia, with partners from both countries participating. The program supports cross-border cooperation in entrepreneurship, innovation, environment, and tourism, involving collaboration between businesses as well as residents of border areas.

Objective

In 2022, LevelLab and the Estonian Entrepreneurship University of Applied Sciences Mainor conducted a study aimed at assessing the achievement of the Estonia-Latvia Programme’s result indicators – the progress of the program is regularly monitored. The study results revealed that the majority of the assessed result indicators exceeded the target level set for 2023. The greatest increase was in the interest and readiness for cross-border cooperation among new businesses, which increased by 15% compared to 2021. There was also a significant increase in the indicator of solidarity among residents of Valga and Valka.

Methodology

During the study, three online surveys and two street surveys were conducted on both the Estonian and Latvian sides. Through the online surveys, representatives of businesses and institutions were interviewed, while through the street surveys, residents of the region were interviewed. For all surveys, proportions of different respondent profiles were predefined in advance. This means that it was determined which region and which gender and age groups of people would be surveyed. The list of respondents for one survey was provided by the client, while for the others, a random sample was taken according to the criteria provided by the client. The questionnaires for all surveys were provided by the client, and the same survey has already been conducted in 2014, 2015, 2018, and 2021.

Project contracting institution: State Support Services Center
Project implementers: LevelLab and the Estonian Entrepreneurship University of Applied Sciences Mainor
Duration: 12/2022 ‍– 02/2023

Assessment of Training Needs for Local Government Officials and Employees

Objective

The aim of the study was to enhance the methodology for assessing the training needs of local government officials and employees, and to conduct an analysis of training needs, including proposals for addressing issues in the development of local government personnel over the next 5-10 years.

Results

2015. In 2015, a pilot study was conducted to map the competencies of local government officials and employees and assess training needs, during which a competency model was developed. The updated and enhanced competency model developed as part of this study consists of three competency groups: general competencies, general competencies in local government management, and sector-specific professional competencies.

The increasing flexibility in working hours and the use of telecommuting necessitate the development of ICT skills, while also increasing the need to enhance employees’ communication and collaboration skills.
The projected training budget for local government authority institutions collectively until 2030 is 24 million euros. This constitutes approximately 1.7% of the total local government employee salary fund.

There is a need for a central training coordination system to be established for the development of local government employees, aiming to coordinate training activities in key priority areas within local governments.

The methods used in the study: surveys, interviews, focus groups, document analysis, and expert panels.

Project ordering institutions: Ministry of Finance and Riigi Tugiteenuste Keskus
Project implementers: LevelLab (lead partner), Baltic Research Institute, and Geomedia
Duration: 09/2020 ‍– 04/2021

View the final report

Analysis of the impact and effectiveness of the Youth Guarantee support system implementation

Background

2018. In 2018, there were 12,400 young people aged 15-24 and 13,200 young people aged 25-29 in Estonia who were not in education, employment, or training (NEETs), accounting for 11.7% of the 15-29 age group. In 2017, the Ministry of Social Affairs introduced the Youth Guarantee Support System (hereinafter NGTS), aimed at identifying young people who are not studying or working (NEETs) and providing them with targeted support from local governments, primarily to continue their education and/or enter the workforce.

Objective

The aim of the study was to assess the relevance, impact, effectiveness, cost-effectiveness, and sustainability of the NGTS implementation.

Results

It was found that overall, the NGTS is effective, and its implementation along with support measures has significantly helped young people to better transition into employment and/or education. The study confirmed that active labor market measures, networking, personalized counseling, and prevention play a significant role in supporting continued education and entry into the labor market. Therefore, considering the low awareness among young people, NGTS, along with other services, directly supports the alleviation of the situation of NEET youth in Estonia.

Methodology

Personal interviews and focus group interviews, document analysis, and quantitative data analysis were conducted.

Project contracting authority: Ministry of Social Affairs
Project implementers: LevelLab and Baltic Institute of Research
Duration: 04/2020 ‍– 12/2020

View the final report

Evaluation of the Effectiveness of the ASTRA Research and Development Program

Background

The Institutional Development Programme for Research and Development Institutions and Universities (ASTRA) is one of the key measures alongside core funding and mobility support in implementing the Research and Development (R&D) and Innovation (RDI) Strategy “Knowledge-Based Estonia 2014-2020” as well as several other national strategies. The objective of the measure is to support structural changes in research and development institutions and universities, as well as the development of institutional responsibility areas and core activities. In total, 17 different universities and research and development institutions have received support from the ASTRA measure for the implementation of 36 projects. The total funding amount is 136 million euros.

Objective

The aim of the study is to assess the effectiveness, impact, and efficiency of the ASTRA measure, and to describe the lessons learned from its implementation.

Methodology

The study primarily employed qualitative methods such as analysis of documented information, interviews, and focus groups with representatives of stakeholders and project implementers.

Benefits

The results, conclusions, and recommendations from the evaluation of the ASTRA measure and the activities implemented within it help prepare new measures for the next structural fund period, including measures that assist Estonia (primarily the Ministry of Education and Research) in planning and supporting the institutional development of research institutions and universities.

Project contracting authorities: Ministry of Education and Research, Estonian Research Council
Project implementers: Baltic Studies Institute (lead partner) and LevelLab
Duration: 04/2021 ‍– 01/2022

View the final report

Overview of Estonia’s Research and Innovation System

Objective

The purpose of the background report was to provide a comprehensive overview and assessment of Estonia’s research and innovation system to the European Commission. The overview included the structure and organization, management and funding, development, and other aspects of the research and innovation system. An assessment was provided on the management of the Estonian research and innovation system, the funding system for research and higher education, and external financing over a period of 10 years.

Main results

  • The most significant driver of economic development and growth is innovation, particularly innovation based on research and development (R&D).
  • The capabilities necessary for creating innovations are needed for the national innovation system to learn, develop the necessary human capital, stay abreast of international advances in science and technology, and maintain its productivity and competitiveness.
  • Innovators typically do not innovate alone but within national and international innovation systems that involve multiple participants and institutions. Therefore, policymakers must ensure that the combination of policies related to various parts of the system is balanced and that interventions across the entire innovation system are coordinated.

Methodology

Quantitative data and analysis, as well as document analysis, were used in the preparation of the background report.

Project contracting authority: European Commission
Project implementers: Technopolis Group Estonia and LevelLab
Duration: 10/2018 ‍– 02/2019

View the final report

Interim Evaluation of the Implementation Plan for EU Cohesion Policy Funds

Objective

The purpose of the interim evaluation was to assess the relevance, effectiveness, impact, and sustainability of the implementation plan and the activities carried out within it. The evaluation covered all 14 priority areas of the implementation plan.

Results

The interim evaluation concluded that the sectoral objectives and activities of the implementation plan remain relevant today, and there is no need for significant changes to achieve the objectives of the implementation plan.

As of December 31, 2018, 32% of the €4.6 billion from the EU Cohesion Policy implementation plan for 2014–2020 has been disbursed, totaling €1.5 billion. The support received from the EU Structural and Investment Funds has clearly contributed to achieving the objectives of all priority areas: the funding has a direct impact on increasing employment, improving public health, developing the living environment, and achieving other objectives. The impact of Structural Funds is long-term and manifests in changes at various levels. The operational program has had a significant impact on achieving Estonia’s key strategic objectives.

Challenges and Recommendations

Regional development balance remains a significant challenge. Overall, regional disparities remain significant, and there has been no noticeable shift towards more balanced territorial development. Close to 50% of EU funds have been invested in Harju and Tartu counties, leading to a widening of regional disparities. EU funds should be directed towards preventing problems and developing comprehensive solutions in regions, with an increased focus on the role, awareness, and competence of local governments in regional investments.

Additional funds from the performance reserve should be directed towards preventive measures and value-added activities such as research and development, innovation in entrepreneurship, education, and regional development.

Research methodology: focus group interviews, individual interviews, workshops, document analysis (documentation of projects, measures, and implementation plans), and quantitative analysis (financial, output, and outcome indicators of measures).

Project contracting authority: Ministry of Finance
Project implementers: Technopolis Group Estonia (lead partner), Ernst & Young Baltic, Praxis, Baltic Institute of Research and LevelLab
Duration: 11/2018 ‍– 04/2019

View the final report

Analysis of the Implementation of the Qualifications System

Background

The qualifications system is part of the Estonian qualifications system – it connects the education system with the labor market and contributes to the development, assessment, recognition, and comparison of work-related competencies. The implementation of the qualifications system has been regulated by the Vocational Qualifications Act since 2001, and the system consists of four parts: the qualifications framework, vocational standards, awarding of vocational qualifications, and the vocational register.

Objective

The aim of the study was to determine the expectations of the participants in the qualifications system, their readiness for implementing the reform, and the transition to a skills-based approach, as well as to identify the potential challenges and risks associated with the planned innovations.

Methodology

During the study, the qualifications system used in Estonia was compared with the experience of other countries. Interviews and focus groups were conducted with direct and related stakeholders of the qualifications system to assess the readiness and expectations of the participants:  applicants for vocational qualifications, issuers of vocational qualifications, educational institutions, learners, stakeholders involved in the development of vocational standards (including employers and professional associations), career counselors, qualifications system specialists, and policymakers. During the study, risk assessment was also conducted to ensure the realism and feasibility of the innovations.  

Benefits

As a result of the study, input was obtained for the renewal of the legal system underlying the implementation of the qualifications system and for the implementation of a skills-based qualifications system.  

Project contracting institution:  Ministry of Education and Research
Project implementer: LevelLab
Duration: 03/2023 – 10/2023

View the final report