knowron_reportifai

• 

  The technical overview of our project

• 

  This is our team

Inspiration

Report writing is an important task in many industries. Especially Blue Collar workers who mostly work off the desks have to set aside valuable time for documentation purposes. According to our sources, blue-collar workers spend up to 25% on repetitive tasks, including documentation 1. Physicians spend even 35%, only, on documentation and writing reports 2. This is incredibly valuable time we want to give back to the workforce.

Equally important is the language barrier, which prevents easy collaboration between coworkers. In 2022 there was a shortage of around 630,000 skilled workers in Germany 3. This can also be due to the language barrier. We are in need of new, skilled people, and the current status quo is excluding many potential talents.

In the age of AI, there should not be an excuse to not use these powerful tools to enable the integration of a strong workforce and simplify repetitive, documentation processes.

• Report creation is one of the main obstacles for highly skilled foreign blue collar workers to be competitive in the german work market.
• At the same time writting monotonous reports takes up a large portion of their working time and hinders their productivity
• Finally, language barrier between coworkers and their supervisor is one of the most important causes of misscommunication and even potential conflicts

What it does

ReportifAI provides a simple, clean, and clutter-free chat interface to document your working steps. The automated translation of an input language into your target language enables much easier and faster communication of workers with each other and with supervisors. The web interface enables the input of text, audio as well as pictures for documentation purposes. The inserted information is automatically summarized and filed in a report. When finishing all steps in a task, the report is generated as a PDF.

- Povides an intuitive, automated solution for filing the work reports

How we built it

ReportifAI is mainly based on Python, with a combined Front- and Backend of Plotly Dash. When inputting an audio file, the OpenAI Whisper speech-to-text API transcripts the spoken words into text. From here the pipeline is the same. The text is translated into English, where it is processed by the OpenAI ChatGPT 3.5 API. Clever Prompt engineering enables us to ask the user for exactly the information we need and display relevant information inside the UI. All responses by the AI get translated back into the input language. When the report information gathering is finished, the AI sends it as a JSON code to the PDF generator, which prints the PDF with all necessary information and the corresponding company logo.

• To ensure the best multilingual user experience → decision to use an extensive and well established language model - ChatGPT
• To make the user experice most pleasant and smooth - minimalistic chat interface (insperation from well know and familiar to everyone interfaces of WhatsApp, …)

Challenges we ran into

Using the ChatGPT 3.5 API turned out to be more complex than simply using the web version of ChatGPT. We had to write our own memory retention so that the AI model does not forget previous information.

A likewise complicated endeavor was sorting the analyzed information into categories such as a very short summary to show on the side of the screen, a list of tasks done as well as a description of what was done during the working day.

Accomplishments that we're proud of

We are very proud of working so consistently and efficiently together as a team. Working through the weekend went by like a flash and we achieved a great result. Not only do we have a fully functioning interface, we use three AIs to enable our users a flexible way of data entry while breaking through language barriers and building bridges between employers and employees.

What we learned

We learned that when using the right tools for the job, amazing results can be achieved even in a short time. Also, we saw that implementing even simple features takes much more time than anticipated.

What's next for knowron_reportifai

After finishing the Makeathon there sure are a few more features to implement! It certainly would be interesting to follow the idea further in programs such as Xplore or maybe even try to build a start-up out of it.

answer the following headlines • What it does? -> 1 sentence • How we built it? ->2 sentence • Accomplishment we are proud of? -> 1 sentence

Sources:

Built With

• cahtgpt3.5
• langchain
• openai
• python
• wisper