Developing Feedback Surveys for the Estonian Academy of Security Sciences

Background

The Estonian Academy of Security Sciences needs to update its principles and procedures for collecting and considering feedback; the current process does not support the goal of improving the quality of education as desired. There are too many feedback surveys, they are conducted in different environments, and the analysis of the results is not consistent or evidence-based.

Objective

The goal is to develop a comprehensive feedback system and update the feedback surveys aimed at learners. The results of the work serve as the basis for the development and implementation of a new feedback collection and evaluation procedure for the Estonian Academy of Security Sciences.

Methodology

To develop the solution, a business analysis will be conducted, including interviews. Subsequently, prototypes of questionnaires will be developed, followed by the methodology for surveys, including a process description, research calendar, analysis methodology, technical implementation of data processing, and visualization solution.

Benefits

The developed solution supports the goal of improving the quality of education, being part of a broader internal and external assessment system. It will be web-based (secure and user-friendly), allowing clear visualization of results and their dissemination to different target groups. Participation in surveys will be limited to students of the Estonian Academy of Security Sciences.

Project contracting institution: Estonian Academy of Security Sciences
Project implementer: LevelLab
Duration: 12/2023 ‍– 03/2024

Developing Feedback Surveys for the Estonian Academy of Security Sciences

Background

The Estonian Academy of Security Sciences needs to update its principles and procedures for collecting and considering feedback; the current process does not support the goal of improving the quality of education as desired. There are too many feedback surveys, they are conducted in different environments, and the analysis of the results is not consistent or evidence-based.

Objective

The goal is to develop a comprehensive feedback system and update the feedback surveys aimed at learners. The results of the work serve as the basis for the development and implementation of a new feedback collection and evaluation procedure for the Estonian Academy of Security Sciences.

Methodology

To develop the solution, a business analysis will be conducted, including interviews. Subsequently, prototypes of questionnaires will be developed, followed by the methodology for surveys, including a process description, research calendar, analysis methodology, technical implementation of data processing, and visualization solution.

Benefits

The developed solution supports the goal of improving the quality of education, being part of a broader internal and external assessment system. It will be web-based (secure and user-friendly), allowing clear visualization of results and their dissemination to different target groups. Participation in surveys will be limited to students of the Estonian Academy of Security Sciences.

Project contracting institution: Estonian Academy of Security Sciences
Project implementer: LevelLab
Duration: 12/2023 ‍– 03/2024

The processing tool for admission statistics at the Estonian Academy of Security Sciences

Background

The Estonian Academy of Security Sciences wanted to implement an automated tool for processing admission statistics queries in the SAIS information system, which simplifies the processing of admission statistics, presenting them as compact data tables and visualizations.

Objective

The aim of the proposed solution was to create an Excel workbook, which takes as input structured data tables (for example, .csv extracts from the SAIS admission information system) or PowerQuery data queries from SAIS.

Methodology

The structure and functionality of the solution were designed during the requirements gathering process, which involved document analysis and interviews with representatives from the Estonian Academy of Security Sciences. During the interviews, the structure of the output worksheets, the formatting of tables, charts, and diagrams, as well as other necessary information, were clarified.

Benefits

The tool helps reduce the time spent on compiling and analyzing admission statistics at the Estonian Academy of Security Sciences, thereby ensuring a faster and more efficient process.

Project contracting institution: Estonian Academy of Security Sciences
Project implementer: LevelLab
Duration: 10/2023 ‍– 12/2023

Prevention program “Puhas Tulevik” effectiveness study

Background

Since 2019, the number of offenses committed by minors in Estonia has been on a declining trend, but the number of drug-related crimes has increased. In order to more effectively change the behavior of minors, a multifaceted approach has been implemented, preferring means of influence over punishments and implementing social programs. Among these is the prevention program “Puhas Tulevik” aimed at reducing drug use and promoting the social and psychological well-being of young people. However, considering the complexity of prevention work and the inadequacy or harmfulness of several measures, it is important to analyze the causes of delinquency, develop and implement evidence-based measures to prevent drug use among youth, and assess the impact and evidence-based nature of the measures already in place.

Objective

The aim of the study is to analyze the evidence-based nature and the current implementation of the prevention program “Puhas Tulevik”, including based on a comparative mapping of local municipalities.

Methodology

For evaluating the effectiveness of the prevention program “Puhas Tulevik,” various methods are employed. Document analysis is used to gather preliminary information and define the theoretical framework.  A survey will be conducted among child protection specialists and local police officers in municipalities implementing the prevention program to map activities and obtain their assessment among program implementers. An analysis of program completers and dropouts will also be conducted. Semi-structured in-depth interviews will be conducted with representatives from four target groups: participants in the prevention program, their parents, local police officers, and municipality officials. The aim is to obtain detailed input to assess the effectiveness of the program  and identify various factors that influence its effectiveness. In the nationwide mapping of the program, a logic intervention analysis is conducted in different municipalities. Additionally, a validation seminar is conducted to present the research findings, gather initial feedback, and validate the conclusions and recommendations of the study.

Benefits

During the study, the aim is to assess whether the activities implemented within the program support the achievement of objectives in the best possible way, and to identify the main challenges in implementing the program. As a result of the study, practical recommendations for further development of the program will be presented, along with suggestions for evaluating the effectiveness of the program in the future.  

Project ordering institution: Police and Border Guard Board
Project implementer: LevelLab
Duration: 11/2023 ‍–09/2024

The startup ecosystem of Ida-Viru: looking back and forward

Background

Startup Estonia implemented a three-year pilot program in Ida-Viru County aimed at creating a favorable environment for the development of startups in the region. The program involved 23 organizations and attracted nearly 21,000 participants. Significant interest and the registration of 14 new startups indicate the region’s potential. The follow-up project for the years 2022-2025 focuses on sustaining the entrepreneurial environment in Ida-Virumaa, aiming to create more new startups and promote collaboration with the creative industries sector.

Objective

The aim of the study is to gather necessary input regarding the current state and challenges of the Ida-Viru startup ecosystem, which could be used for planning future ecosystem development activities. Evaluation of the impact and quality of the project activities since 2019.

Methodology

The work will involve using multiple data collection and analysis methods. Document analysis will help assess the impact and quality of the previous pilot project, providing input for evaluating the current status and future development opportunities of the Ida-Viru startup ecosystem. Surveys will assess the awareness and readiness of Ida-Viru residents for entrepreneurship, and analyze the views of participants in events organized by startup support organizations on the Ida-Viru startup ecosystem. Focus group interviews will be conducted with providers of all types of services (such as incubators, accelerators, coworking spaces, and event/conference organizers) as well as representatives of startups operating in various sectors.

Benefits

The results of the study will help to understand the current situation and challenges in the Ida-Viru startup ecosystem and provide input for planning and improving future development activities and projects.

Project contracting authority: Estonian Ministry of Economic Affairs and Communications
Project implementer: LevelLab
Duration:10/2023 ‍– 03/2024

Evaluation of the Performance-Based Round for Labour Market Services

Background

Population aging is currently the main demographic trend in both Estonia and Europe, which, coupled with deepening labor shortages, requires effective measures to integrate older adults into the workforce. Through European Union support measures, the Results-Based Call Measure 3.2 was implemented, focusing on enhancing the employability of individuals aged 55 and older. Prior to announcing the call for proposals, the funding scheme for supported activities targeting the older age group was also amended, implementing simplified reimbursement methods – fixed-sum payments and standardized unit costs.

Objective

The aim of the study was to assess the relevance, effectiveness, impact, cost-effectiveness, and sustainability of the approach/funding scheme used, and to provide recommendations for organizing call for proposals in the future.

Methodology

The study employed both qualitative and quantitative data collection and analysis methods, including document analysis, quantitative data analysis, interviews, and a validation seminar. The purpose of the interviews was to gather in-depth input for evaluating the relevance and effectiveness of the program. The interviewees included grant recipients, project participants, representatives from the Ministry of Social Affairs, and representatives from the State Support Services Centre.

Benefits

As a result of the study, the justification for the results-based approach in open call tenders for employment policy implementation was identified, and suggestions were made for future improvements in organizing results-based grant schemes in the field of employment policy.

Project ordering institution: Ministry of Economic Affairs and Communications
Project executor: LevelLab
Duration:04/2023 ‍– 12/2023


Analysis of the Implementation of the Qualifications System

Background

The qualifications system is part of the Estonian qualifications system – it connects the education system with the labor market and contributes to the development, assessment, recognition, and comparison of work-related competencies. The implementation of the qualifications system has been regulated by the Vocational Qualifications Act since 2001, and the system consists of four parts: the qualifications framework, vocational standards, awarding of vocational qualifications, and the vocational register.

Objective

The aim of the study was to determine the expectations of the participants in the qualifications system, their readiness for implementing the reform, and the transition to a skills-based approach, as well as to identify the potential challenges and risks associated with the planned innovations.

Methodology

During the study, the qualifications system used in Estonia was compared with the experience of other countries. For assessing the readiness and expectations of stakeholders in the qualification system, interviews and focus groups were conducted with direct and related stakeholders of the qualification system: applicants for certification, certifying bodies, educational institutions, learners, stakeholders involved in developing occupational standards (including employers and professional associations), career advisors, qualification system specialists, and policymakers. During the study, risk assessment was also conducted to ensure the realism and feasibility of the innovations.

Benefits

As a result of the study, input was obtained for the renewal of the legal system underlying the implementation of the qualifications system and for the implementation of a skills-based qualifications system.

Project contracting institution: Ministry of Education and Research
Project implementer: LevelLab
Duration: 03/2023 – 10/2023

View the final report

Evaluation of the Effectiveness of the ProgeTiigri Program Activities

Background

The ProgeTiiger program was launched in 2012 in order to develop students’ digital competence and technological literacy and to understand the nature of technology, its application, and development opportunities. The program’s resources have supported the development of teachers’ digital competence, promoted networks of teachers and instructors in the field, and co-financed the acquisition of equipment necessary for teaching.

Objective

The aim of the study was to assess how the ProgeTiigri program has contributed to the improvement of the quality and accessibility of technology education from 2018 to 2021. In addition, recommendations were provided for modifying program content activities and for follow-up actions.

Methodology

The study utilized both qualitative and quantitative data collection and analysis methods (document analysis, surveys, focus group and individual interviews). The survey involved teachers, educational technologists, and educational institution leaders from Estonian general education schools. Personal interviews were conducted with employees of the Ministry of Education and Research, the Education and Youth Board, and the ProgeTiigri program managers. The focus group was conducted with representatives and experts from universities, as well as with educational leaders and/or teachers actively involved in the program.

Benefits

As a result of the study, feedback was provided on the activities of the ProgeTiigri program during the period 2018-2021, and an assessment was made of how purposeful the activities have been and what improvements have been made in the quality and accessibility of technology education. Additionally, recommendations were provided on what changes could be made to the content activities of the program and how to proceed with the program.

Project contracting institution: Education and Youth Board
Project implementer: LevelLab
Duration: 01/2023 ‍– 05/2023

View the final report

Assessment of Training Needs for Local Government Officials and Employees

Objective

The aim of the study was to enhance the methodology for assessing the training needs of local government officials and employees, and to conduct an analysis of training needs, including proposals for addressing issues in the development of local government personnel over the next 5-10 years.

Results

2015. In 2015, a pilot study was conducted to map the competencies of local government officials and employees and assess training needs, during which a competency model was developed. The updated and enhanced competency model developed as part of this study consists of three competency groups: general competencies, general competencies in local government management, and sector-specific professional competencies.

The increasing flexibility in working hours and the use of telecommuting necessitate the development of ICT skills, while also increasing the need to enhance employees’ communication and collaboration skills.
The projected training budget for local government authority institutions collectively until 2030 is 24 million euros. This constitutes approximately 1.7% of the total local government employee salary fund.

There is a need for a central training coordination system to be established for the development of local government employees, aiming to coordinate training activities in key priority areas within local governments.

The methods used in the study: surveys, interviews, focus groups, document analysis, and expert panels.

Project ordering institutions: Ministry of Finance and Riigi Tugiteenuste Keskus
Project implementers: LevelLab (lead partner), Baltic Research Institute, and Geomedia
Duration: 09/2020 ‍– 04/2021

View the final report

Evaluation of the Effectiveness of the ASTRA Research and Development Program

Background

The Institutional Development Programme for Research and Development Institutions and Universities (ASTRA) is one of the key measures alongside core funding and mobility support in implementing the Research and Development (R&D) and Innovation (RDI) Strategy “Knowledge-Based Estonia 2014-2020” as well as several other national strategies. The objective of the measure is to support structural changes in research and development institutions and universities, as well as the development of institutional responsibility areas and core activities. In total, 17 different universities and research and development institutions have received support from the ASTRA measure for the implementation of 36 projects. The total funding amount is 136 million euros.

Objective

The aim of the study is to assess the effectiveness, impact, and efficiency of the ASTRA measure, and to describe the lessons learned from its implementation.

Methodology

The study primarily employed qualitative methods such as analysis of documented information, interviews, and focus groups with representatives of stakeholders and project implementers.

Benefits

The results, conclusions, and recommendations from the evaluation of the ASTRA measure and the activities implemented within it help prepare new measures for the next structural fund period, including measures that assist Estonia (primarily the Ministry of Education and Research) in planning and supporting the institutional development of research institutions and universities.

Project contracting authorities: Ministry of Education and Research, Estonian Research Council
Project implementers: Baltic Studies Institute (lead partner) and LevelLab
Duration: 04/2021 ‍– 01/2022

View the final report