Inspiration
Our team consists of one Electrical Engineering, one Computer Science, and two Mechanical Engineering first-year students. Wanting to play to our team’s strengths, we decided to compete under the hardware mechathon track with a web application implementation to supplement the robot. As mostly beginner codes and engineers, we wanted to create a project that could help solve real-world problems and have meaningful impact while also being feasible within 36 hours.
When we heard the initial John Deere presentation, we were shocked to hear that 20% of all United States worker deaths are from the construction industry as one of the most dangerous careers. This is especially sad since many of the tragic accidents could have been easily prevented with the right application of technology.
After further research, our team learned about the significance of level grounding, and the dire consequences of unlevelness. Tractor overturns are #1 cause of death in the agricultural industry. According to the National Institute for Occupational Safety and Health, around 200 farmers a year were fatally crushed when machines flip. Unlevel grounding is the primary reason for these heavy equipment tipovers.
The implications of unlevel grounding goes beyond this. Unlevel land wastes water, and causes up to 30% of loss due to runoff. This also causes uneven irrigation and agricultural foundations. Leveling is very important as it is the only way to achieve precision for maximum yield and transportational efficiency for food production.
Inspired by John Deere's legacy of solving these exact problems, we built the Sandozer: a sand-leveling robot that combines a bulldozer chassis with a flat blade, equipped with an array of sensors and backed by an interactive web application (Mission Control) that visualizes, monitors, and communicates everything the robot experiences in real time.
What it does
We modified the Arduino Elegoo Owlbot Tank Kit Robot to have two main components: a “rough cut” leveler in the front and a smoothing plate in the back. Through the actuated arm in the front, the teeth of the leveler roughly evens out the surface in front of the robot, prepping the area for being smoothed by the plate behind. Our robot has a front wheel drive to aid in turning and comes fitted with various safety mechanisms, like tire guards and safety sound effects. This allows us to easily flatten sand or dirt and traverse different types of terrain.
The Mission Control web application has four distinct modes the user can directly interact with. The navigation mode accepts custom path commands either typed or spoken (like f f r f l f) and animates the robot’s journey step-by-step across the grid. While doing so, there is narration of the sensor readings at each gric cell using the ElevenLabs AI voice. The test mode allows the user to enter in any designation coordinate on the grid and the application uses breadth-first search to calculate the shortest path, then animates the traversal with full sensor readouts. The use-CV mode uses a Google Teachable Machine computer vision model to classify terrain type from either desert, forest, mountain, or plains from a live webcam feed or an uploaded image. The chat mode has an AI assistant powered by Gemini API which is able to answer questions about the robot, its sensors, past missions, and navigation capabilities. This assistant can also pull context from the Snowflake mission history database.
Beyond this, there are multiple other features of the web application. Powered by the Snowflake database, the application uses machine learning to classify hazards depending on the mission’s tilt, terrain, AQI, and path data. The model uses data-driven safety intelligence and learns from previous sensor readings to predict whether a given tilt angle and terrain combination poses a real tip-over risk.
Additionally, the model has a real-time safety system with boundary proximity alerts, tilt risk indicators, air quality warnings, and a configurable traction profile (dry packed, dry loose, or wet/moist sand). There is voice control embedded throughout the entire application where users can speak commands, path directions, coordinates, or chat questions hands-free, as well as an emergency stop button on all the robot traversal screens.
How we built it
The mechanisms of our robot first began with prototyping and ideation. After looking towards real life bulldozers and motor graders, we developed the concept of having a slightly slanted plate out front to push the sand to be more even while having a plate behind to smooth over any rough patches from the wheels. Based on the [what came in the kit], our robot is two wheel drive, specifically the front wheels, with a separate motor controlling each wheel. This allows for forward and backwards motion as well as turning motion without the need to develop a robust steering system. The actuating arm in the front is controlled by one large servo in the middle to distribute force evenly. The arms of the leveler are also relatively short in order to minimize the torque necessary from the servo to lift up or down. The plate in the back is not actuated and is able to freely pivot in order to adapt to the incoming surface. The goal of the back plate is not to make the surface level, but to smoothen it out, as leveling the surface is the job of the front arm.
The web application is built with HTML/CSS/Javascript, with a custom .tech domain from the MLH support. We used Cloudflare to streamline the deployment of the website. Specific notable algorithms include the BFS pathfinding implemented from scratch for shortest-path navigation and the use of SVG path rendering to draw the robot’s trail dynamically on the grid as it moves. We used the Google Teachable Machine software with TensorFlow.js for real-time computer vision in an in-browser experience. For voice control, we used the Web Speech API for input and the ElevenLabs API for AI voice narration. The Gemini API powers the chatbot, trained especially on the Sandozer. We used Snowflake as a mission archive database that saves every completed traversal’s path, tilt, AQI, and timestamp. We also build ML inference engine using Snowflake’s build-in SQL ML functions to classify terrain bumps as hazardous or non-hazardous from mission data. Lastly, we used the Cloudflare Key-Value vector storage for Retrieval-Augmented Generation (RAG) to let the chat assistant reference real past missions.
Challenges we ran into
Accomplishments that we're proud of
We are very proud to have a presentable product! We have implemented safety features, and this system mirrors real construction standards with boundary alerts, tilt-based terrain classification, separate designations for different types of sand, and simulated air quality readings. The entire app is voice-operable and hands-free, which matters in heavy equipment contexts and industrial, real-world situations. The CV runs with zero latency and we have created a cohesive system for obtaining, storing, and analyzing data, as well as making predictions based off of this! Despite being a hardware hackathon, the web app genuinely looks and functions like professional mission control software, with lots of care taken for the UI/UX experience.
What we learned
Throughout this mechathon, we learned a lot about the importance of planning and prototyping, overcoming persistent challenges, and exercising creativity when problem solving. A lot of our time throughout the mechathon involved designing and redesigning mechanisms as we learned more about the shortcomings of our initial designs in practice. While the process of iteration and prototyping is the foundation of the engineering process, a lot of time could have been saved if we had built a more robust prototype earlier on in the challenge, rather than designing and manufacturing entire systems only to have to redesign it completely a few hours later. Throughout this challenge, we also faced a multitude of recurring problems, from components not integrating as intended to the Elegoo Arduino Nano not “synching” with our computers. This meant a lot of time spent frustrated in trying to solve the same issue and oscillating back and forth between solutions. Ultimately though, we were able to overcome every one of our major setbacks, with the frustrating process resulting in a sweeter victory in the end. Due to constraints with time, there were a lot of components, specifically linkages, that were not able to be as custom fit as we would have liked. However, with the many laser cut components we had from our iterative design phase, we were able to find interesting connections not originally intended by the linkage. This allowed for some major overhauls in the leveling arm and drive system later on in the challenge to be completed even without any manufacturing abilities.
What's next for Sandozer
Our next steps for the sandover is to fully automate the robot! Currently, we are only able to semi-automate the robot through use of the ‘sand breakers’ which break apart large pieces of sand. Additionally, we also have an ultrasonic sensor which stops the robot from moving if it ‘sees’ an object. However, we would move on to having a fully autonomous control system. This would require a greater number of sensors on the robot which we did not have enough time to implement as well as a deeper dive into the software. Furthermore, we would also like to see a more complete integration between the Arduino robot and the Mission control application: for example, this could been as a real-time location of the robot on the ground as well as being able to ‘stop’ and ‘start the robot from the application in the event of a mishap. Finally, we aim to add an air quality sensor and fan to the arduino robot. The air quality sensor would measure the amount of sand in the air due to the robot’s movements and could inform the driver if there is high agitation. Furthermore, adding a fan on the top of the arduino robot would allow the user to more easily even the sand from a greater distance.
Log in or sign up for Devpost to join the conversation.