Partalyzer is great for sorting out a pile like this!
Take pictures of the chip with your phone
Define the chip boundries
A picture of the whole chip
Cropped chip photo showing the text
Chip surface after processing with OpenCV
Output of Tesseract when run with the processed image
Partalyzer Inventory Management Homepage
The Partalyzer app returns the datasheet after querying the backend with the part number
Data visualization of Resistors by value using Google Charts
When discussing ideas for this hackathon, our team realized that we had brought a lot of spare ICs that we had collected over the years from various personal projects, but we couldn't remember what they did! It would be nice to just be able to point your phone camera at an IC, resistor or capacitor and know what it is, how it works, and just how many are in our university's stock room...
What it does
Partalyzer uses an Android app to take a photo through your phone and crop the resulting picture to include the surface of the device containing the part number. This picture is then sent to a backend server that uses OpenCV to perform raw image processing (e.g. filtering) and Tesseract to perform OCR and extract the part number as text. This text is then checked against a local database to see if it is in the organization's stock and has a locally cached datasheet link. If the datasheet is not found, the backend queries Octopart, an electronics component database, and retrieves a datasheet link. Currently, manual entry of part numbers is fully supported and OCR is only partially supported.
How we built it
- Android App: We wanted a simple interface which allows users to take images of their components and interact with their inventory. What better platform than a smartphone, which has a camera built in! The app lets users select some cursory information about a device, such as category (resistor, capacitor, IC, etc.), and then allows them to take an image of the component. The image is passed to a server for further processing and handling. A secondary interface allows the user to manually enter part information in the case that the image cannot be processed. For IC's, the app waits for the server to return a link to a datasheet, which it launches right in the browser. All the information you need right at your fingertips!
- Part Number OCR: The key to turning a picture into a part number is image processing. We used images from a number of phones to develop an adaptive thresholding algorithm that can turn a picture of a chip into a black and white, 'flattened' image. With this processed image, we run Tesseract optical character recognition (OCR) to extract the text on the chip label. Some post-filtering is performed on this text, removing invalid characters, datestamps, and logos. Finally, a query is sent to a backend database with the presumed part number of the photographed chip.
- Backend Database: A MySQL database is used to store a log of all items that are scanned. This includes the time of scan, scanning user's ID, category of the part, the value of the resistor/capacitor (or model number of IC), and quantity.
- API: An API is used to send messages between the Android app and the backend. The API allows the app to submit components to the database, and additionally allows the app to query the database.
- Web Interface: A web interface, which utilizes the above API, displays information from the database about which parts are in inventory. The interface utilizes a responsive Bootstrap theme, which makes it usable on a variety of platforms including mobile. The quantity of the parts is summed based on its category and value (or model number) and displayed in a searchable DataTable. There are tabs for each component type: Resistor, Capacitor (Electrolytic) and Capacitor (Ceramic), IC.
- Analytics: On the web interface, statistics about the inventory on hand can be viewed. This information allows stakeholders to make decisions about when to re-order components (Low Stock), the distribution of parts by category (Inventory Visualization), and how parts have been used over time (Individual Component Visualization).
- Octopart Query: Octopart has a RESTful API that allows for querying and returning information about part numbers. The backend server attempts to match the OCR text to an existing part number in the local organization's database and then sends Octopart a Manufacturer Part Number (MPN) query. This is to ensure that the datasheet query is using a part number that exists within the Octopart database. If the local part number corresponds to a part number within Octopart's database, the backend then performs a query to obtain a URL for the part datasheet. This URL can then be sent to the client's phone for reading.
Challenges we ran into
- OCR is difficult with no background experience...
- Hardware camera APIs in Android were difficult because of limitation of phone hardware...
Accomplishments that we're proud of
- We were able to successfully extract text using OCR at a good enough quality to perform queries in the local and Octopart databases and return the datasheet for ICs to the client phone.
What we learned
- Image processing is hard but cool!
- Python is a powerful tool!
- Cell phone cameras aren't the best for OCR
What's next for Partalyzer
- Obtain hardware capable of consistently taking photographs at the proper quality for performing OCR
- Add graphical representations of statistics such as part stock quantity history
- Expand Octopart API usage to return comparison of current vendor prices for ordering