We were inspired by PhotoMath and what it represented for mathematics, and we thought it might be useful to have such a tool for schematics. Moving the Schematics from paper form to digital means that you have the freedom to use it and store it anywhere. Easier integration with BMS (building management systems) Maintenances and assets management software. Easier facilities management and control

The user interface was based around WolframAlpha's, because we're striving to be a PhotoMath for schematics.

What it does

It recognizes the schematics from a pics and allows the user to see digital connexions within them. The user can export the schematic as a JSON or GraphViz file. The user can also see the schematic as a manipulable graph.

How we built it

We used OpenCV for the recognition and VueJS for the UI. We do line and symbol detection then recognize the connections and the letters. This greatly simplifies the recognition task.

Challenges we ran into

Recognition of lines (connections) was hard because if you only segment lines, you get all lines, also the ones of other symbols. Furthermore only color-based extraction was also not possible, because in the hand-drawn images there were black lines inside the red ones, which made segmentation impossible, so we used some kind of thresholding over all RGB-Channels in the image to extract the connection lines.

Integrating Cytoscape into Vue

This was difficult because the model used by Vue was very different from Cyto. It took several hours to get them to play together, even with the vue-cytoscape plugin and involves jerry-rigging some CSS to guarantee the positioning is correct. This was probably trickier than trying to integrate AppKit stuff into SwiftUI.

Accomplishments that we're proud of

  • Getting DigiMatics up and running semi-remotely with a new team in very little time.
  • The quality of the user interface and the care given to the æsthetics

What we learned

  • Breaking up of the problem into smaller pieces and having a MVP
  • Using Vue.js and Vuex.

What's next for DigiMatics

We think the next thing to pursue would to miniaturise our vision stack and try to implement it on a smartphone or tablet, so that it may be used on the go. Apart from that, we also think it would be interesting to generalise this solution, so that other schematics (e.g. electrical, gas) may be used. Another thing to consider would be allowing natural language queries to be processed on the graph (e.g "is pump X connected to valve Y") which would simplify any necessary cross-checking.

Built With

Share this project: