The challenge: our inspiration and who we are
Ever happened to you, that you dramatically looked at mobile apps’ privacy policies without understanding anything? “How will my data be handled? May I trust them?” This would be even more true with the spreading of COVID-19 related apps. TRAP4C would answer these concerns, placing transparency at the centre of its actions. Yet it aims to help developers as well. They struggle to find integrated legal & technical guidance, considering the increasing number of guidelines and recommendations, to navigate the nuances of COVID-specific privacy issues.
We are an international, interdisciplinary, and dynamic group of PhD fellows in Law and Technology (Last-JD-RIoE PhD Program), within a European doctoral consortium led by the University of Bologna. We decided that we were ready to raise up to the challenge.
Our solution and its impact on the future
TRAP4C is the solution that COVID19 app developers and consumers have been waiting for. Our team built an interactive webpage through which app developers can self assess the compliance of their app with state of the art data protection and e-health legislation through an easy questionnaire. At the same time, citizens can compare and contrast several apps through a user-friendly interactive webpage. This webpage clearly shows how a COVID app scores in terms of transparency, fairness and lawfulness through bar charts and other intuitive graphic elements. Ethics, Law and Technology together: it’s real.
TRAP4C is an app that brings a simple yet elegant solution to a problem that existed even before. Citizens and app developers alike were already confused by the complex and multi-sided legal framework concerning personal data and the relationship between fundamental rights and technology. Covid-19 amplifies these concerns. TRAP4C can be used as a privacy compliance tool even after the end of the Covid-19 outbreak. TRAP4C is not a checklist and not even a simple transparency report. It allows citizens and developers to learn about privacy in a practical yet complete way.
How we built it
The technical challenge of this project is to build a user-centred explanatory tool for a self-assessment process that involves complex legal knowledge about privacy and data protection. The technology behind our explanatory tool was partially supported by the European Union's Horizon 2020 research and innovation programme under the MSCA grant agreement No 690974 “MIREL: MIning and REasoning with Legal texts”.
In order to address user-centrality, explanations need to cater to the individual in terms of her/his background, context, need and purposes. A single explanation output regardless of all these backgrounds and needs (i.e., a One-Size-Fits-All explanation) would become massive and unwieldy as soon as the data and processes to be explained become averagely complex, and the readers approaching such explanations would be immediately overwhelmed by the sheer size of the data.
This is why we designed an interactive explanatory tool. With our explanatory tool, we show an example of a new approach to explanations that build over a clear separation between explainability and explaining, with the goal to collect and organize explainable information, articulating it into user-centred Explanatory Narratives (EN). Our working definition of EN takes inspiration from epistemology literature, integrating concepts of usability defined in ISO 9241-210.
Through the use of explanatory narratives we represent the problem of generating explanations for complex decision processes (such as the self-assessment process) into the identification of an appropriate path over an explanatory space, allowing explains to interactively explore it and produce the explanation best suited to their needs. In this specific scenario, the explanatory space has been defined by a conceptual map of relevant information to explain the self-assessment process to an EU citizen.
Through the use of our user-centred explanatory tool, we empower the user (the citizen, or the developer) to interactively explore legal content about EU privacy and data protection legislation thus gathering Descriptive, Teleological and Justificatory explanations of the underlying complex process.
The formulation of a specific COVID 19-related questionnaire also involved considerable technical expertise of the legal Team. The questions were built around the most recent guidelines of EU bodies on the use of COVID 19-related apps, which further highlights the novelty of the project. Complex legal content was mapped and rephrased in an accessible and user-friendly fashion. This passage also required intensive work of the legal Team’s members, who built a comprehensive knowledge graph of privacy and data protection legal concepts.
The TRAP4C web app is at a prototype stadium of development. So far, we carried out an evaluation of the user-centred explanatory tool. Our hypothesis was that, through our user-centred explanatory tool, usability (effectiveness, satisfaction and efficiency) is improved. We specifically designed a usability test of the user-centred explanatory tool applied to a system unrelated to the self-assessment process, and also simpler.
In order to verify our hypotheses, we tested our explanatory software on an explainable AI-powered credit approval system using the FICO Explainable Machine Learning Challenge dataset and we probed into it from the perspective of different users. We compared a static explanatory tool for post-hoc explanations named Contrastive Explanations Method (CEM), with an interactive version built using our model for user-centred explanatory tools.
We conducted a User-Study with 58 participants (28 males and 30 females; age: 18-77) from the UK, US and Ireland. Half of the participants were asked to test the static explanatory tool while the other half was asked to test our interactive explanatory tool. Results indicated that the user-centred tool we designed is likely to be more useful than the static explanatory tool. We then performed one-sided Mann-Whitney U-tests (MW) under the alternative hypothesis that the user-centred system (V2) is more usable than the other one (V1), meaning that the resulting effectiveness, satisfaction and efficiency is greater.The results show that:
- effectiveness score improvement in V2 is significantly greater than V1 (U=320.0, p=0.04)
- satisfaction score improvement in V2 is significantly greater than V1 (U=318.0, p=0.05) Overall, the results show that even at a prototype stadium, the designed web app has great potential in terms of usability.
Our Added Value
We add transparency and trustworthiness to the apps to ensure their wide usability. We bring the power back to the public- taking into their own hands their data protection through the publication of self-assessments in a clear manner.
What we have achieved during the Hackathon weekend
During the Hackathon weekend, we firstly concluded the Q&A for the self-assessment of COVID-apps developers. Then, we built the “model self-assessment tool” by bridging together the legal, ethical and technical frameworks. Afterwards, we developed and structured the legal knowledge base into a conceptual map and at the same time, we worked on the front-end and on the back-end of the prototype platform. We also finalised the video, the pitch, and elaborated a feasible business plan.
What we need to keep on the right track
We are looking for excellent, driven front-end developers to help us in scaling the project. We are looking for assistance with the set-up of the scoring system. Finally, we also would like to translate the website into different languages.