Still nowadays, humans and machines do not collaborate effectively together. Allianz showed us that even cracks at micro-level can have a huge impact. Sixt is fighting the problem of personell-intensive assessment of damages to their mobility assets. For example, the risk of giving decisions about repairs only to machines would be too high.

So we combine the best of both worlds, our excellent Human-Decision-Making capabilities with the raw and tremendous power of Machine-Learning in the Cloud. We seek to improve the currently slow and exhausting process of data classification, by giving the possibility to interact.

What we provide

„Human-in-the-Loop“ Training for complex classification tasks providing a usable system from the start. We continuously receive images of cracks or other features from various cameras, which will be evaluated. Hard classification entries are forwarded to a human decision maker. The decision maker’s feedback will in turn improve the ML-model and help future classifications. A convenient, easy-to-use and integrateable user interface is provided for user interactions (like Slack or Mattermost).

How we built it

We built a docker-based microservice solution that consists of 5 parts:

  1. The image acquiring system (camera)
  2. The Core (acts as an interconnect for all the other services and handles database)
  3. The ML-Connector (is responsible for training & modeling ML)
  4. The Mattermost-Connector (handles the connection to the user-interface)
  5. Mattermost (provides the user-interface)

Since it is a Microservice application, we were able to develop the different services using different programming languages, for example the Mattermost-connector and the AI-Connector in Golang, the Core in Node.JS and the camera adapter with Python. For the ML-tasks, we use the Azure CustomVision, but it would be possible to connect to any ML framework.

Challenges we ran into

  • Clearly defining interfaces between services using REST
  • One huge challenge was the connection to Azure (bugs in the official Golang-SDK lead us to building creative work-arounds)
  • Microservice orchestration of different independent services

Accomplishments that we’re proud of

  • Providing a working prototype.
  • Having a at least moderately capable AI model
  • Each of us was able to use his favorite programming language but still working together on one project.

What we learned

  • Communication is key, as with microservices as with humans
  • A little bit of golang for everybody
  • Python glues everything together
  • Docker enables heterogenous developer teams
  • Sleep is for the weak!

What's next for Autocrack

  • Support for more types of messengers
  • Improvement of AI Backend
  • Improvement of the AI models

Stay hyped for our release!

Share this project: