• 

  architecture

• 

  design

• 

Ojas - Contactless Heart Rate Monitoring

🫀 Inspiration

During a late-night conversation with my sister, she mentioned how uncomfortable traditional heart rate monitors felt during her daily health checks. That got me thinking - what if we could measure vital signs without any physical contact? I'd recently learned about remote photoplethysmography (rPPG) and it clicked: your smartphone camera can detect the subtle color changes in your face caused by blood flow!

The challenge that really motivated me was the Arm AI Developer Challenge. I wanted to push the boundaries of what mobile processors could do - not just make another app but build something that truly leverages ARM's specialized hardware like Neon SIMD and Neural Processing Units. It became personal: could I create a medical-grade app that runs entirely on-device, with no cloud dependency, that's actually usable in real-world scenarios?

💡 What it does

Ojas (Sanskrit for "vital energy") transforms your Android phone into a heart rate monitor - no wearables, no contact, just your camera and some clever mathematics.

Here's the magic:

1. Point your camera at your face - MediaPipe Face Landmarker detects 468 facial landmarks in real-time
2. We sample specific regions - Forehead and cheeks are optimal for detecting blood volume changes
3. Extract the green channel - Hemoglobin absorbs green light differently as blood pulses through capillaries
4. Signal processing wizardry - ARM Neon-optimized FFT finds the dominant frequency (your heartbeat!)
5. AI refinement - TensorFlow Lite with NNAPI cleans up motion artifacts using the ARM NPU

The result? Accurate heart rate readings in under seconds, with real-time signal quality feedback and a beautiful, animated interface that feels like something from a sci-fi movie.

Key Features:

• ✅ Real-time HR monitoring (45-180 BPM range)
• ✅ Live signal quality assessment ("Excellent", "Good", "Fair", "Poor")
• ✅ Animated heartbeat indicator that pulses with your actual heart
• ✅ Visual waveform display
• ✅ No internet required - 100% on-device processing
• ✅ Battery efficient

🛠️ How we built it

This was a masterclass in squeezing every drop of performance from ARM hardware. Here's the architecture:

The Vision Pipeline (Kotlin + MediaPipe)

// CameraX streams at 30 FPS (640x480)
// MediaPipe Face Landmarker runs in LIVE_STREAM mode
faceLandmarker.detectAsync(mpImage, timestampMs)

// Extract ROI (Region of Interest) from forehead + cheeks
val greenSignal = extractGreenChannel(bitmap, landmarks)

The Signal Processing Core (C++ with ARM Neon)

This is where things get intense. I wrote a custom signal processor in C++ that uses ARM Neon SIMD intrinsics for 4x speedup:

#ifdef USE_NEON
    // Process 4 floats simultaneously using ARM Neon
    float32x4_t sumVec = vdupq_n_f32(0.0f);
    for (int i = 0; i < vecSize; i += 4) {
        float32x4_t dataVec = vld1q_f32(&data[i]);
        sumVec = vaddq_f32(sumVec, dataVec);  // Vectorized addition
    }
#endif

The pipeline:

1. Circular buffer (300 samples ≈ 10 seconds)
2. Normalization (mean subtraction + std-dev scaling) - Neon optimized
3. Hamming window (reduces spectral leakage)
4. KissFFT (Fast Fourier Transform to frequency domain)
5. Peak detection (find dominant frequency in 0.75-3.0 Hz range)
6. Frequency → BPM conversion ($HR = f \times 60$)

The AI Layer (TensorFlow Lite + NNAPI)

Training a lightweight 1D CNN was crucial for cleaning motion artifacts:

model = tf.keras.Sequential([
    Conv1D(16, 7, activation='relu'),  # Multi-scale feature extraction
    MaxPooling1D(2),
    Conv1D(32, 5, activation='relu'),
    Conv1D(64, 3, activation='relu'),
    GlobalAveragePooling1D(),
    Dense(32, activation='relu'),
    Dense(1)  # HR output
])

Then converting to TFLite with FP16 quantization for ARM NPU acceleration:

converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.float16]

The NNAPI delegate configuration ensures it runs on the ARM Cortex-M NPU:

nnApiDelegate = NnApiDelegate(
    NnApiDelegate.Options().apply {
        setAllowFp16(true)              // Enable half-precision
        setUseNnapiCpu(false)           // Force NPU usage
        setExecutionPreference(
            EXECUTION_PREFERENCE_SUSTAINED_SPEED
        )
    }
)

The UI (Jetpack Compose)

I wanted something that felt alive. Every element responds to your actual heart:

// Heartbeat animation synced to actual BPM
LaunchedEffect(bpm) {
    while (true) {
        beat = true
        delay(100)
        beat = false
        delay((60000f / bpm - 100).toLong())
    }
}

😅 Challenges we ran into

1. The Great Neon Nightmare

My first attempt at ARM Neon optimization crashed spectacularly. Why? I was using 32-bit ARM flags (-mfpu=neon, -mfloat-abi=softfp) on 64-bit ARM (arm64-v8a). Spent 6 hours debugging before I realized ARMv8 doesn't need -mfpu - it's always enabled! The CMake fix:

if(ANDROID_ABI STREQUAL "arm64-v8a")
    target_compile_options(ojas PRIVATE -march=armv8-a)  # No -mfpu!
    target_compile_definitions(ojas PRIVATE USE_NEON)
endif()

2. The Blue Screen of Doom

At one point, the app just showed a solid blue screen. No logs, no crashes, nothing. The culprit? I'd forgotten to load the TFLite model into assets! The PulseML initialization was silently failing. Added proper error handling:

try {
    val modelBuffer = FileUtil.loadMappedFile(context, "rppg_model.tflite")
    interpreter = Interpreter(modelBuffer, options)
    Log.i(TAG, "✅ Model loaded successfully")
} catch (e: Exception) {
    Log.e(TAG, "❌ Model loading failed: ${e.message}")
}

3. Face Tracking... or Lack Thereof

MediaPipe's callback wasn't receiving bitmaps. Turns out I needed to extract the bitmap manually from the MPImage:

// WRONG
.setResultListener { result, input ->
    handleResult(result, input.bitmap)  // bitmap is null!
}

// CORRECT
.setResultListener { result, input ->
    val bitmap = BitmapExtractor.extract(input)
    handleResult(result, bitmap)
}

4. Signal Quality Issues

Initial readings were all over the place. The problem? I wasn't handling the circular buffer correctly in C++. When converting buffer indices to a linear array for FFT, I had an off-by-one error that corrupted the signal. Fixed with:

for (int i = 0; i < bufferSize_; i++) {
    int idx = (writeIndex_ + i) % bufferSize_;  // Proper circular indexing
    fftInput_[i] = buffer_[idx];
}

5. Battery Life

Running camera + face detection + FFT + ML inference was draining 15% battery per minute! Optimizations that saved the day:

• Reduced camera resolution from 1080p → 640x480 (3x reduction)
• Only compute HR every 1 second (not every frame)
• Use STRATEGY_KEEP_ONLY_LATEST to drop frames when processing is slow

🏆 Accomplishments that we're proud of

1. It actually works!

Testing against a pulse oximeter, Ojas achieves ±15 BPM accuracy on a good signal. That's honestly better than I expected for a first prototype.

2. Performance metrics that matter:

• 30 FPS camera pipeline (consistent)
• <10ms TFLite inference with NNAPI
• <5ms signal processing per computation
• ~50ms total latency (essentially real-time)
• <5% battery drain per minute

3. Legitimate ARM optimization:

This isn't fake optimization. The code has:

• ✅ 3 Neon-vectorized functions (mean, std-dev, normalization)
• ✅ Explicit Neon intrinsics (vld1q_f32, vaddq_f32, vmulq_f32, etc.)
• ✅ NNAPI delegate properly configured for ARM NPU
• ✅ FP16 quantization for mobile efficiency
• ✅ CMake flags proving compilation with -march=armv8-a

4. A beautiful, functional UI:

I'm genuinely proud of the interface. The pulsing quality indicator, the synchronized heartbeat animation, the live waveform graph with color-coded quality - it feels professional and polished.

5. Signal quality analysis:

Beyond just heart rate, the app computes:

• Signal-to-noise ratio (SNR)
• Motion artifact detection
• Periodicity verification
• Overall quality assessment

This gives users confidence in their readings.

📚 What we learned

Technical Lessons:

1. ARM Neon is powerful but tricky: You need to understand the difference between ARMv7 (32-bit) and ARMv8 (64-bit) architecture. The flags and intrinsics are different!

2. NNAPI isn't magic: Just adding the delegate doesn't guarantee NPU usage. You need:

   • Compatible operations
   • Proper quantization (FP16)
   • Explicit preferences
   • Fallback handling

3. FFT on mobile requires care: KissFFT is great because it's lightweight, but you need proper windowing (Hamming) and normalization or your frequency spectrum is garbage.

4. MediaPipe is amazing but opaque: The documentation is... sparse. Trial and error was necessary for the callback structure and bitmap extraction.

Soft Skills:

1. Debugging patience: Some bugs took hours to find. The blue screen issue taught me to add logging everywhere during development.

2. Scope management: I wanted to add HRV analysis, export to PDF, breathing guides, multi-user profiles... Had to ruthlessly prioritize what would actually demo well.

3. Performance vs. accuracy trade-offs: Could I get better accuracy with a larger model? Yes. Would it run at 10 FPS and kill the battery? Also yes. Learning when "good enough" is actually optimal was key.

🚀 What's next for Ojas

Short-term (next month):

1. Heart Rate Variability (HRV) analysis - Already prototyped! HRV is a better stress indicator than raw HR. Formula:

$$\text{SDNN} = \sqrt{\frac{1}{N-1} \sum_{i=1}^{N} (RR_i - \overline{RR})^2}$$

where $RR_i$ are inter-beat intervals.

1. Export & Share - Generate PDF reports with historical data, graphs, and trends

2. Breathing guide - Visual pacer for box breathing (4-4-4-4 pattern) to improve signal quality

Medium-term (3-6 months):

1. Blood Oxygen (SpO₂) estimation - Research shows it's possible with red+green channel ratio analysis

2. Stress level tracking - Using HRV metrics + resting HR to estimate autonomic nervous system state

3. Wear OS version - Smartwatch with always-on display showing continuous HR

Long-term (dream features):

1. Blood pressure estimation - Extremely challenging, but some research papers show promise using PTT (Pulse Transit Time)

2. Integration with health platforms - Google Fit, Apple Health (via companion iOS app)

3. Clinical validation study - Partner with a medical institution to validate against ECG gold standard

4. Expand to elderly care - Remote monitoring dashboard for caregivers, with anomaly detection and alerts

---

🎓 Technical Specifications

For the judges reviewing ARM optimizations:

Requirement	Implementation	Evidence
ARM Neon SIMD	3 vectorized functions	signal_processor.cpp lines 61-158
Compilation Flags	-march=armv8-a, -O3, -ffast-math	CMakeLists.txt lines 25-30
Neon Intrinsics	vld1q_f32, vaddq_f32, vmulq_f32, vsubq_f32, vdivq_f32, vst1q_f32	Full usage in mean/stddev/normalize
NNAPI Acceleration	Explicit delegate configuration	PulseML.kt lines 36-51
FP16 Quantization	setAllowFp16(true)	Confirmed in logs
Target Architecture	arm64-v8a only	build.gradle.kts

Performance Benchmarks:

• Signal processing (Neon): 4.2ms vs CPU scalar: 16.8ms (4x speedup)
• TFLite inference (NNAPI): 8.9ms vs CPU: 47.3ms (5.3x speedup)
• Total pipeline: ~50ms end-to-end latency

---

💻 Try It Yourself

GitHub: https://github.com/namdpran8/Ojas

Requirements:

• Android 8.1+ (API 26+)
• ARM64 device (Snapdragon 660+, Exynos 9610+, Dimensity 700+)
• Front-facing camera
• Good lighting

Quick Start:

git clone https://github.com/namdpran8/Ojas
cd ojas
./gradlew assembleDebug
adb install app/build/outputs/apk/debug/app-debug.apk

---

Built with ❤️ (literally measuring it!) for the ARM Device Solutions Hackathon.

"The best way to predict the future is to invent it... and then measure your heart rate while doing so." 🫀

Built With

• c++
• kotlin
• mediapipe
• python