Inspiration
Our inspiration was to turn an iPhone's built-in LiDAR sensor into a real-time object tracking and collision prediction system, as a small demonstration of how LiDAR can be used as a precollision detection system and how it can be more commerically used within space travel..
What it does
Our Debris Collision Detector is a proof-of-concept system that uses an iPhone's LiDAR sensor, camera, and gyroscope to detect and track nearby objects in 3D space. The application streams sensor data to a computer, where our software analyzes object movement, estimates trajectories, and predicts potential collisions.
Our system can:
- Display live camera and LiDAR depth information.
- Track objects and estimate their movement over time.
- Predict future positions and identify potential collision risks.
- Estimate the closest approach between a tracked object and the simulated spacecraft.
- Compare possible evasive movements, including moving left, right, up, down, forward, or backward.
- Display tracking information and suggested avoidance maneuvers through a live web dashboard.
Although our prototype operates at short distances rather than detecting actual orbital debris, it demonstrates the fundamental concepts behind autonomous collision avoidance.
How we built it
We built our system around an iPhone 18 Pro, using its LiDAR scanner, camera, and gyroscope as our primary sensors.
Initially, we experimented with an Arduino Uno and existing sensor-streaming applications. However, we realized that sending sensor data directly to a computer would simplify the system and give us considerably more processing power, and reduce possible handshake issues via usbc.
We to a custom iOS application developed with Swift and Xcode, using ARKit for depth information and device positioning, alongside motion sensor data and we developed a Python-based processing system that receives sensor information, tracks objects across frames, estimates their trajectories, and calculates potential collision risks.
We also built a browser-based dashboard that visualizes sensor data, tracking results, and possible avoidance maneuvers.
To improve performance, we redesigned the processing pipeline to prioritize the newest available sensor frames rather than allowing outdated frames to accumulate. We also introduced higher-frequency tracking and prediction updates to reduce latency.
Challenges we ran into
One of our biggest challenges was establishing reliable communication between the iPhone and computer. Existing streaming applications created compatibility problems, especially when switching between operating systems and trying to connect through USB or a mobile hotspot.
Another major issue was latency. Initially, the phone produced sensor updates faster than our tracking system and dashboard could process them. This created a noticeable delay between an object's actual movement and its displayed position.
We also had to work through the complexities of combining LiDAR depth measurements, camera images, and device motion into a consistent coordinate system.
Finally, predicting collisions proved more complicated than simply detecting objects. We needed to estimate movement over time, account for relative motion, and determine which evasive direction would provide the greatest predicted clearance.
Accomplishments that we're proud of
We're especially proud of developing a system that combines consumer-grade sensors with real-time trajectory prediction and collision avoidance concepts.
Some of our biggest accomplishments include:
- Building a computer-based LiDAR visualization and object-tracking pipeline.
- Creating a dashboard that combines live sensor information with predicted object trajectories.
- Developing an avoidance algorithm that evaluates multiple possible movement directions instead of simply warning about collisions.
- Improving the tracking pipeline to reduce delays and prioritize fresh sensor data.
- Achieving promising results in simulated 30–60 FPS tracking tests, with initial prediction latency reduced from approximately 0.6 seconds to 0.23 seconds in our synthetic benchmarks.
- Developing and testing a custom sensor receiver as part of our move toward native iOS streaming.
Most importantly, we turned an ambitious idea into a working software prototype with testable detection, tracking, prediction, and avoidance components.
What we learned
This project taught us how much goes into building a real-time sensing system.
We gained experience with LiDAR depth sensing, ARKit, Swift, Python, network communication, and object tracking. We also learned how sensor update rates, processing delays, and data synchronization can significantly affect prediction accuracy.
One of our biggest takeaways was that collecting data is only half the challenge. Processing that data quickly enough to make useful decisions is equally important.
We also developed a better understanding of trajectory prediction, relative motion, and the tradeoffs involved in designing autonomous collision avoidance systems.
What's next for Debris collision detector
Our next priority is improving the speed, accuracy, and reliability of the system.
We plan to finalize our native iOS application and validate direct USB communication to minimize latency and eliminate dependence on third-party streaming applications.
We also want to improve object tracking during rapid movements, incorporate more advanced motion prediction algorithms, and make our avoidance recommendations more accurate.
Eventually, we hope to introduce simulated orbital debris environments with realistic velocities and trajectories, allowing us to test our algorithms under conditions closer to those encountered by spacecraft.
Our long-term goal is to expand Debris Collision Detector from a LiDAR-based demonstration into a more sophisticated autonomous collision prediction and avoidance platform.
