Inspiration
The rapid advancements in autonomous vehicle technology have underscored the importance of robust object detection and tracking systems. Simulators like CARLA offer a controlled environment to develop and test these systems. By integrating YOLOv9 for object detection with DeepSORT for tracking, this project aims to enhance real-time multi-object tracking capabilities, contributing to safer and more efficient self-driving technologies.
What it does
This project combines YOLOv9 and DeepSORT to detect and track multiple objects in real-time within the CARLA simulator. The system identifies various objects, such as vehicles and pedestrians, assigns unique identifiers, and monitors their movements across frames. This continuous tracking provides valuable data for applications like trajectory planning and decision-making in autonomous driving scenarios.
How we built it
Installed CARLA 0.9.11 and configured the environment for simulation.
- Data Collection: Utilized the trajectory_planning.py script to simulate autonomous vehicle navigation within CARLA, generating and recording video data of dynamic environments.
- Object Detection with YOLOv9: Integrated the YOLOv9 model, fine-tuned for detecting relevant objects in the simulated environment.
- Object Tracking with DeepSORT: Applied the DeepSORT algorithm to associate detected objects across consecutive frames, maintaining consistent tracking IDs.
- Integration and Visualization: Developed the detect_dual_tracking.py script to process recorded videos, performing detection and tracking, and generating output videos with annotated bounding boxes and tracking information.
Challenges we ran into
- Synchronization Issues: Ensuring synchronized data collection between the CARLA simulator and the detection-tracking pipeline was challenging, requiring precise timing mechanisms.
- Performance Optimization: Balancing real-time processing demands with the computational load of running both YOLOv9 and DeepSORT necessitated optimization of code and efficient resource management.
- Environmental Variability: Handling diverse scenarios within the CARLA simulator, such as varying lighting conditions and object occlusions, posed challenges for maintaining detection and tracking accuracy.
Accomplishments that we're proud of
- Seamless Integration: Successfully combined YOLOv9 and DeepSORT into a cohesive pipeline that operates efficiently within the CARLA simulator environment.
- Real-Time Performance: Achieved real-time object detection and tracking, processing video at an average of 24 frames per second on standard hardware configurations.
- Robust Tracking: Maintained high tracking consistency, with DeepSORT successfully tracking 92% of detected objects across 1,000 frames, ensuring reliable object identification and monitoring.
What we learned
- System Integration: Gained insights into integrating multiple complex systems, ensuring seamless data flow and processing between the CARLA simulator, YOLOv9, and DeepSORT.
- Performance Tuning: Learned techniques for optimizing performance in real-time applications, including efficient resource utilization and code optimization strategies.
- Handling Simulation Variability: Developed strategies to manage and adapt to environmental variability within simulations, enhancing the robustness of detection and tracking algorithms.
What's next for YOLOv9 and DeepSORT for Self-Driving Cars
- Enhanced Object Classification: Expand the system to classify a broader range of objects, including cyclists, traffic signs, and road markings, to provide more comprehensive environmental understanding.
- Trajectory Prediction: Integrate predictive algorithms to anticipate the future positions of tracked objects, aiding in proactive decision-making for autonomous vehicles.
- Real-World Deployment: Transition from simulation to real-world testing by applying the developed system to data collected from actual driving scenarios, assessing performance and making necessary adaptations.
- Community Collaboration: Encourage collaboration by sharing the project's codebase and findings with the research community, inviting contributions to further enhance the system's capabilities.
Log in or sign up for Devpost to join the conversation.