The Challenge
15-year-old legacy systems
In 2020, Statistics Estonia started a procurement process for developing an application to replace two major systems currently used for collecting data from businesses and private individuals in Estonia. Both legacy systems had been developed over a 15-year period and were outdated technically and design-wise.
Read also: Solutions for legacy software modernization
Voluntary participation in surveys
While companies are required by law to submit certain data about their activities for statistical purposes, it entails no direct benefit for them. Participation in surveys is voluntary for private persons, and the question of motivating people to participate in surveys is even more warranted. When respondents don’t fill in the surveys online, employees of Statistic Estonia need to contact them individually, which is resource-costly and slows down the process of obtaining statistics.
Meet the customer
e-Estonia – Building a digital society
The Estonian dream is to have as little government as possible, but as much as is necessary. Thanks to e-solutions, communicating with the state is fast and convenient, and our country is more effective.
Named ‘the most advanced digital society in the world’, ingenious Estonians are pathfinders, who have built an efficient, secure, and transparent ecosystem that saves time and money.
The Goal
Simplify the data collection process and increase user satisfaction through a new data collection system.
Rather than update the existing software, Statistics Estonia decided to rethink the entire data collection process and replace the legacy systems with a single state-of-the-art application.
The organization had one big overarching goal with the new application – to make the process of submitting data as simple and user-friendly as possible, which would motivate greater participation and lessen the workload of employees who’d otherwise have to contact the participants individually by mail or phone to get the necessary information.
Consequently, user experience was a key criterion when planning and prioritizing development tasks throughout the project.
Work Scope
It was estimated that building the whole system would take place over a four-year period. The first phase was to be completed by the end of the first year, so the system could be used for carrying out the first national population census in a decade. This goal meant a tight deadline for the first year of development since the census date was fixed; the census would take place at the end of 2021 and focus on the conditions of December 31st.
As it was a fixed-scope project, the main requirements had been decided and set down during the procurement process. However, building the system from scratch meant a lot of liberty, but also responsibility, in designing the architecture of the solution. The data collection application was also the first project to incorporate Statistic Estonia’s new CVI, which presented another interesting challenge; we had no previous guidance to rely on but knew our designs would be used as a reference for future projects.
The Process
If you are familiar with Helmes, you know that we follow a specific collaboration process with clients.
Read more about the Helmes Way.
The development period started with our customary kickoff meeting, a day-long session with all Helmes and Statistics Estonia team members. We specified the most important requirements, defined metrics, and agreed on priorities. The development phase itself was divided into two- to three-week sprints; this meant continuous analysis and integration and regularly shipping new features to the client for review and testing.
The Team
On the client side, the team included five key members who were regularly involved in the process. On the Helmes side, we had on board a team lead, an analyst, a tester, and multiple developers – a fairly common mix of roles for a software project. However, because the user experience was paramount in the project, we also enlisted extensive help from our internal service design team.
The service/UX designers from the Helmes service design team not only helped us with their expertise in industry standards and best practices but also with mapping the existing situation and the prospective ideal solution, interviewing various stakeholders who would be using the system, carrying out user tests, and analyzing feedback.
Investing so heavily in the service design is not always part and parcel of the development process, especially in the case of public services, so we’re glad that Statistics Estonia accepted our proposal to dive deep into the service design aspect of the project. And we can say the client was very satisfied with the results.
Statistics Estonia selected Helmes as their best collaborator
The Solution
The most important part of the project is creating the new data collection application. The national census is just one of the services on the platform.
The new application was built from scratch
The data collection application is an online platform built from scratch using Java and open-source technologies. The system has two views: external users (respondents) and internal users (employees of Statistics Estonia).
GDPR-compliance
When an external user (the respondent) first logs onto the platform, they will see the businesses they are legally affiliated with. They can manage authorizations for the company and give access to other persons, for example, such as employees who will be filling in the surveys on their behalf. They will also see any surveys they have been attributed to.
Survey functionality
The internal user’s view involves more functionalities. The employees can create surveys and send them to designated audiences, follow the progress of the surveys and see the results. Selecting respondents (samples of companies/individuals who need to fill in the survey) and detailed analysis of results are done externally with other specialized tools.
Prefilled data from state databases
After logging in, the respondents can provide all the requested information on the platform. Whenever possible, the data is prefilled – Statistic Estonia has access to over 30 state databases and uses register-based information when it is available.
No duplication needed
If there is a need to obtain data from a respondent over the phone or in person, the interviewer can fill in the data in the same application, so there is no need for duplication. This can even be done offline. The application is also to be used to communicate with the respondents to notify them about their surveys and deadlines, send reminders, etc.
The Launch
Estonian Population Census of 2021
The data collection application developed by Helmes got its first baptism of fire in late 2021 when Statistics Estonia carried out the first nationwide population and housing census in a decade. The purpose of the national population and housing census is to collect information about the composition, location, and other aspects of a country’s population. In EU Member states, a census is coordinated by Eurostat, the statistical office of the European Union.
Innovative method used
For the first time, the 2021 Estonian population census was carried out using the combined method (i.e., using existing information from state databases and requesting from people only data that is not available in registers). In addition to the main registers – the population register, the registers of the Land Board, the Tax and Customs Board, and the Health Insurance Fund – information from more than 30 state databases was used in preparing the census.
The central principle in practice
As most of the information was prefilled using data from various registers,
respondents only needed to answer a small number of questions – for instance, regarding their nationality, language skills, religious beliefs, and household composition.
The Result
Thanks to the new approach and integration of nearly 30 state databases, the average time required to complete the survey online was cut drastically – from 51 minutes to 5 minutes – compared to the 2011 census. According to Statistics Estonia, 568,891 people, or 43 percent of the Estonian population, completed the 2021 population census online, which meant a combined time savings of 54,519 work days.
The time savings will compound even further when considering the reduced work from the staff of Statistics Estonia, who will interview a sample of the remaining population.
These results are a promising precursor of the future, as from 2024 onwards, all EU countries should carry out a population census every year, eventually transferring to fully register-based censuses.
I am very happy that Statistics Estonia has a partner who takes a broader view of the cooperation and who is prepared to contribute even more in both thought and deed than originally agreed.
Marika Korka, Statistics Estonia’s data collection team lead
Get in touch
