Ensuring data privacy and user-friendly personal information management has been and remains to be a delicate and complicated subject. Governments and the private sector have sought solutions to balance secure personal information management with providing a better service for years.
Even though Estonia has been at the forefront of adopting new technologies quickly and implementing them in providing fast and flexible digital services, there was plenty of room for development in how personal data could be managed and shared, especially between state organizations and the private sector.
Meet the customer
In Estonia, the Information System Authority (Riigi Infosüsteemi Amet) is responsible for developing national IT systems and national cybersecurity. It is the central state authority ensuring that government information systems are secure, operational and connect citizens, government databases, and state-provided digital services.
RIA develops and manages the national data exchange layer X-Road, the administration system for the state information system (RIHA), the secure Document Exchange Layer, electronic identity trust services (eID), data communication in public administration (ASO), and the state portal (eesti.ee). It is responsible for ensuring that the digital state of Estonia is functional, sustainable, and always reachable.
The client was looking for a process and technical solution to provide a universal and transparent consent service for private individuals, which would allow the sharing of personal information between the state and the private sector.
A person would need to have the ability to:
- Give consent to share selected personal information already available in a database to private sector companies;
- Revoke given consent at any time;
- Have an overview of given consents;
- Have an overview of shared data.
A universal consent service makes it possible to share your data in government databases with the private sector. You can choose who you want to share your information with, what information you want to share, and decide to stop sharing whenever you feel like it; this saves you time and means you can get services faster and easier.
This service can be useful when a bank or other credit provider asks for your financial background information or when you ask to receive medical services, obtain insurance, or in many other cases where your information is necessary to provide a personalized product or service.
The project’s objective was to create a publicly accessible open-source consent service solution, which offered a standardized structure and user experience and could be implemented in any government or state database.
This result would open new possibilities for using personal data for scientific research, create an incentive for the private sector to develop innovative services for citizens, and make it possible to offer personalized services by private sector companies.
Getting the job
Even though we had a previous working relationship with the RIA, all significant government developments must undergo a thorough public procurement procedure.
The client was looking for an open-source solution that could be run either centralized or decentralized, would provide a unified user interface, and be accessed through multiple systems. It would also need to be based on a microservices architecture and be highly scalable.
We opted for an approach based on the Netflix stack microservices architecture, which would provide fine-grained services. This design scheme would make the whole consent service highly maintainable and testable, independently deployable, and easily scalable. The Netflix stack would also ensure that the service could be implemented both centralized and decentralized, giving added flexibility in the development stage.
Our approach was the best, and we were selected to develop the solution from more than 10 participants.
We always start with a thorough analysis and development project plan, which keeps all involved parties informed and expectations aligned.
We agreed to proceed with a centralized approach for the initial development during the analysis stage after presenting the pros and cons of centralized and decentralized development approaches. This would keep the development and testing processes more effective and straightforward, ensuring we moved forward faster.
The tight time frame of the project meant we had to use highly flexible tools; we chose JHipster to get the microservices and user management up and running as fast as possible.
As the first stage consisted of a lot of analysis and development of backend data request processes, it took us a couple of months of our regular Scrum sprints to get to a point where we could demo anything tangible to the client’s administrators.
By that time, it had become clear that we would have to rethink our process to be even more flexible if we wished to keep to our set schedule.
Switching up the agile
The project was complex enough, with a lot of different parties on both sides. With personal data management, legal issues are a significant risk factor, so many of these concerns had to be discussed by the client’s legal team and their external consultants at Ernst & Young.
The client was represented by its product owner, who was our go-to contact. But their team also included a system architect and several service managers.
In addition to our five-member core team, we also coordinated with our in-house user experience team and external quality assurance tester.
To manage the information flow effectively, we had daily stand-up meetings, weekly meetings with the client, demos and ongoing Skype chats between our team and the client. All the meetings took place via Skype or Microsoft Teams to save time and keep the distance required to control the ongoing COVID pandemic.
We tailored our workflow to maximize delivery by combining the most suitable elements of the agile project management methodologies Scrum and Kanban. The personalized “Scrumban” approach seemed to work very well for the project’s complexity and managed to keep all the related parties communicating with the agility needed for us to move forward at the required pace.
All the details were documented and accessible to both parties in team organizing tools Jira and Confluence.
The resulting universal consent service will be implemented by several pilots, including a health service provider looking to offer an immunization/vaccination passport, or a bank or other credit provider asking for your financial background information.
A similar consent service can also easily be implemented for the private or non-profit sectors to obtain the consent necessary to offer personalized services and discounts to their employees, members, and stakeholders through third-party providers. Such a service can be used to ask for permission to share information with other companies to offer personalized benefits.