Heartscape

Anxious, and the breathing exercises never stick? Heartscape turns your live heartbeat into colour and music, then guides breathing and AR grounding so you can watch your body calm down.

Comment

Inspiration A lot of us have dealt with anxiety. When you see a GP or a psychologist about it, the first thing they usually give you is homework: slow breathing, or a grounding exercise like 5-4-3-2-1, every day. We never stuck with it, because nothing showed us it was doing anything. So we built an app that lets you see your own heart respond while you practise.

What it does Heartscape transforms your body and the world around you into music and colour. As your heart beats, the screen pulses with colour, which you can personalise using the colour pad to make the experience your own. Meanwhile, your heart rate sets the tempo, while your body's signals and surrounding sounds shape the chords and harmonies, creating a unique soundtrack that evolves with you in real time.

From there you can practise slow breathing (in 4, out 6) or 4-7-8 breathing on top of your own colours, with a voice guiding you and the phone vibrating gently on each breath. Or you can do 5-4-3-2-1 grounding in AR: you look around your room, tap the things you notice, and each one lights up. When you finish, the app shows your heart rate before and after.

Over time the app learns what you like. If you always choose softer colours or turn the drums off when your heart rate is high, it starts doing that for you. All of this stays on your iPhone. There is no account and nothing is uploaded.

How we built it It is a SwiftUI app for iPhone with a watchOS companion. The Watch runs a workout session and sends heart rate to the phone every few seconds. Resting heart rate, HRV and steps come from HealthKit.

The music uses two Core ML models we trained with Create ML. Every bar they look at your heart rate trend, HRV, activity, how loud the room is, the time of day and the colours of a scene, and pick the next chord.

The AR grounding uses ARKit and RealityKit with the LiDAR scanner, and a small Metal shader that sends a ring of light across your room on each heartbeat.

The voice guidance was made with ElevenLabs and is bundled into the app, so it works offline.

Challenges Getting heart rate in real time was harder than expected. Health only updates every few minutes, so we had to stream it from the Watch. We also merged two parts that were built separately over the weekend, the colour and AR app and the music engine. And the phone got hot, so we profiled it and stopped screens from drawing when they were hidden.

What we learned Seeing your own body respond is what makes you want to do the exercise again.

What's next Testing it with people who are seeing a therapist, and letting them share a short summary with their clinician if they want to.

Heartscape is not a medical device and does not replace a GP or psychologist. It helps you do the practice you have already been given.

Built With

  • arkit
  • avfoundation
  • core-haptics
  • core-ml
  • core-motion
  • create-ml
  • elevenlabs
  • healthkit
  • metal
  • realitykit
  • swift
  • swiftui
  • watchos
Share this project:

Updates

Submission history