Drishti

Project Design
Response for Fruit Chart
not fruits Response
Response for rotten fruits

Inspiration

I was discouraged from buying apples at the store when I discovered that one apple was rotten. Due to the way the apples were displayed, it was difficult to inspect each one individually, and I ended up having to discard the entire batch when it became contaminated a few hours later. This experience highlighted the need for a system that can help prevent the spread of contamination in produce and ensure that it stays fresh for as long as possible. I want to create a system that can identify and notify the status of fruits and vegetables in real time, helping to track the quantity of items for replacement and prevent the waste of produce.

What it does

Drishti is a system that uses computer vision to identify and track the status of a variety of fruits and vegetables in real time. The system is able to recognize different types of produce and determine their freshness based on visual cues, such as the presence of blemishes or discoloration. Drishti provides notifications to users when action is needed, such as when produce is approaching its expiration date or when the quantity of an item is running low. The system is designed to help reduce food waste, save money, and improve the efficiency of inventory management. It uses open-source models for privacy and includes a simple user interface for demonstration purposes. Drishti is able to automate prompts for best model search and improve the accuracy of the system over time.

How we built it

We built Drishti through a series of iterative steps, taking the time to carefully identify and address the main issues along the way. Each week, we focused on solving a specific problem and integrating the resulting module into the overall system. To begin, we conducted research on the key challenges in identifying and tracking the status of fruits and vegetables. We discovered that building an evaluation dataset for produce was a major hurdle, as there were limited resources available for this purpose. To overcome this issue, we collected and labeled our own dataset, carefully curating a diverse set of images that represented a range of produce types and conditions. Next, we turned our attention to the task of identifying and tracking the status of the produce. We considered a variety of different approaches, including the use of machine learning models and computer vision algorithms. Ultimately, we decided to use a combination of these techniques, leveraging the strengths of each to create a robust and accurate system. To improve the accuracy of the system, we implemented a number of automation techniques, including the use of Dspy to automate prompts for best model search. We also made a conscious effort to choose open-source models for privacy, as we wanted to ensure that the system was transparent and trustworthy. Throughout the development process, we regularly tested and evaluated the system to ensure that it was meeting our goals. We also made a point of seeking feedback from users and incorporating their suggestions into the design. After several weeks of development, we were able to successfully connect all of the modules and create a working demo of Drishti. We were proud of the progress we had made and excited about the potential of the system to make a real impact in reducing food waste and improving the efficiency of inventory management

Challenges we ran into

One of the main challenges we faced during the development of Drishti was building an evaluation dataset for fruits and vegetables. There were limited resources available for this purpose, and we found that collecting and labeling our own dataset was a time-consuming and labor-intensive process. To overcome this challenge, we carefully curated a diverse set of images that represented a range of produce types and conditions. We took care to ensure that the dataset was balanced and representative, and we implemented a number of techniques to improve the accuracy of the labels. For example, we used a combination of manual labeling and automated tools to annotate the images. We also implemented a quality control process to review and verify the labels, and we made a point of seeking feedback from users to ensure that the dataset was accurate and useful. Despite these efforts, we encountered a number of challenges along the way. For instance, we found that some types of produce were more difficult to identify and track than others, and we had to fine-tune the models to improve their accuracy. We also encountered issues with lighting and perspective, which affected the quality of the images and made it more difficult to extract useful features.