As a health-conscious person, I want to make sure that I am healthy. I want to detect diseases early and to increase my level of wellbeing. Especially in the case of Covid19, I might be in danger myself and also a danger for my community. But I do not want to bother my medical doctor so often to do checkups.


Heartbeat runs as an app on smartphones, leverages existing smart things technology (watches, fitness trackers, scales, ...) to gather data. It integrates into the daily routine of the user, records a short video of an interaction where the user is asked to cough, take a deep breath, say a short sentence and is asked to tell about their impression about health state and symptoms that may have occurred. Heartbeat data, activity data, and other data from smart things is also integrated. The data is analyzed by our central services. As the app is integrated into the daily routine. It can detect anomalies in heartbeat data, face and face movement, the sound of the voice, the sound of breathing or coughing not only on an onetime basis but also on a gradual changes basis. The app calculates a health state rating locally and recommends how the user could improve their health state or also whether the user should see a doctor. The user can override the health state in case of wrong predictions. The system learns to predict the health state better.

A user can share the collected data securely with medical doctors so that they can provide a better diagnosis. The system highlights the symptoms it has detected so that a medical doctor can check anomalies quickly.

We apply the highest standards to ensure the protection of personal health data. Data is made available on explicit consent only and stored encrypted.

Learned improvements in the prediction model for each user will be used to improve the prediction models for all users.

Users will be proud of their contribution to help to control current Covid19 and to prevent future pandemic outbreaks by using the app and providing anonymized health-state data. The daily health state of a user will be sent to the central service anonymously combined with rough information about the user's location. We make sure that it is not possible to identify a person from that data.

Users can decide whether they want to donate personal data (video data, smart things data, ...) to further improve the system, but this is not mandatory.

The central services of Heartbeat analyze the information provided by users of the app, from other data sources like Google search trends, social media monitoring, information from the health care system, open data (weather, travel data) to predict the health state of the population and to identify possible epidemiology relevant events.

The Heartbeat smartphone app is available in a freemium model for users. Persons that just want to check their health state from time to time can use the app for free. The premium version offers extended continuous tracking of their overall health state and recommendations on how to improve it.

The central Heartbeat services provide real-time data about the health state of a population. The data can be used to predict outbreaks of diseases, epidemiology relevant events, or demands for drugs and medical treatments. That kind of information is very valuable and can be sold to governments, healthcare system institutions.

What have we done during the hackathon?

We started from the very scratch, so we had to analyze the idea first and identify all necessary and crucial aspects of the idea:

  • We performed a feasibility study whether it is possible to detect symptoms with smartphones and wearables. We could identify a good amount of research papers from the fields of medical research and AI research. Research results provide evidence that it is possible.
  • We checked how we could collect the necessary data with smartphones and wearables. We checked the APIs of major brands and verified them.
  • We created and validated a prediction model for Covid19, which uses several symptoms such as fever, cough sound types, or breathing problems.
  • We created a business model to monetize the app.
  • We implemented a clickable visualization
  • We created a first design how the final app could look like

Solution’s impact on the crisis

Heartbeat enables persons to self-monitor their health-state on many symptoms, also complex symptoms like cough sounds or ECG anomalies. The app recommends seeing a doctor clarify detected anomalies. So it reduces the number of calls and visits to medical doctors. As users can share the collected data and the analysis of the system with their doctor, they can decide on the next steps much faster. The central services provide anonymized health-states of many users. This data can be used for epidemiologic analysis and predictions.

The value of your solution(s) after the crisis

The app supports health-conscious persons to track and improve their health state. Using the data provided by the central services epidemiologic anomalies can be identified early. Both use cases address a market, which we estimate to ~50m EUR/month in EU, Americas, and APAC.

Necessities in order to continue the project

We plan to do following the next steps after the hackathon:

  • Integrate symptom detection models from research papers into our symptom detector models. It would be necessary to get access to all training and validation data that has been used.
  • Validate prototype with key user groups like medical universities and researchers.
  • Validate the business model

To continue the project on a long term basis we have to find funding to be able to implement and verify all parts of the system. We think of funding options like EU research grants for the research related parts and VC capital for building the company.

Share this project: