Inspiration
What it does
How we built it
Challenges we ran into
Accomplishments that we're proud of
Inspiration
Campus roads are shared by students, pedestrians, cyclists, and vehicles. Even a small obstacle or a person crossing unexpectedly can create a safety risk. I wanted to explore how AI and computer vision could help identify these situations and make campus mobility safer.
That idea led to CampusNav Sentinel — a project focused on helping autonomous campus shuttles understand their surroundings using cameras and AI, without depending on LiDAR.
What It Does
CampusNav Sentinel combines computer vision and AI to analyze road scenes. Its core capabilities include:
- Object detection: Identifies people, vehicles, and other objects using YOLOv8.
- Depth estimation: Uses MiDaS to estimate relative depth from camera images.
- Hazard awareness: Combines visual information to help identify potential road obstacles.
- AI scene analysis: Uses an AI-powered analysis layer to interpret detected objects and provide a clearer summary of the scene.
The goal is to turn camera input into useful information that can support safer campus navigation.
How I Built It
I built CampusNav Sentinel by bringing together computer vision, deep learning, and a lightweight application interface. OpenCV handles image-processing tasks, while YOLOv8 performs object detection and MiDaS estimates depth from images.
I also developed an agentic state machine and a visual-evidence feedback loop to organize the perception workflow and connect observations with scene analysis. FastAPI provides the web interface and connects the different components.
For the AI component, the project is designed to use NVIDIA models through Nebius infrastructure, keeping the system aligned with the hackathon's focus on practical AI applications.
Challenges I Faced
One of the biggest challenges was bringing several AI components together into a single workflow. Object detection and depth estimation have different computational requirements, and combining their outputs requires careful handling of image data, processing time, and dependencies.
Another challenge was designing the system to work with camera-based perception rather than relying on LiDAR. Camera images provide useful visual information, but monocular depth estimation is not a direct replacement for precise physical distance measurements. This makes it important to distinguish between detected objects, estimated depth, and confirmed hazards.
I also had to consider how to make the output understandable instead of simply displaying model predictions. The goal was to connect perception results with a useful explanation of what the system sees.
What I Learned
This project helped me understand how individual AI models fit into a larger application. I gained practical experience working with computer vision, object detection, depth estimation, API integration, and state-based workflows.
I also learned that building an AI application involves more than getting a model to produce predictions. Integration, error handling, clear output, and honest communication of limitations are equally important.
What's Next
My next goal is to improve scene understanding, evaluate the system under different lighting and road conditions, and explore how its outputs could connect to a simulated autonomous shuttle.
CampusNav Sentinel is an ongoing step toward making AI-powered campus mobility safer and more practical. It is a perception and scene-analysis project, not yet a validated autonomous-driving safety system.
Log in or sign up for Devpost to join the conversation.