PineappleHack Sustainable Report

• 

  PineappleHack2023

Inspiration

Purpose & Motivation

The purpose of taking this challenge is to enhance my Azure A.I skill in automated document processing. This time I digged into sustainability report such as ESG Report, etc. I really motivated to apply this solution to unlock the Petronas problem statement. On the other note, I also explored other Azure 's web services product provides the end result of the automated document analysis.

In future, the automated document processing using the A.I capability, is the most looking forward by the business decision maker.The future features of the PineappleHack's app are as below:

To recommend sustainability report recommender solution based on 3 sustainability factors; Environmental, Social Impact and Governance. The ability of the apps to automate and categorizing the company sustainability initiaves based on “Environmental”, “Social” or “Governance” factors. This solution will help the company to decide on future sustainability direction, thus will makes the company to have the clear information on the status of their current green sustainability initiatives.

What it does

Problem Statement

Processing large sustainable documents are really challenging. Decision maker who needs to make investment decision really need to digest a huge chunk of financial reports as such; ESG Report, and other available related sustainable reports such as Annual Reports, Financial Operational Report, Sustainability Disclosure Report and etc. Analysing the sustainable report information will be confusing because it consists a lot of text and number with multiples pages. Sometimes the pages for each company report can go to hundreds of pages. Thus, having an sustainability recommender report will help reduce the human effort in analysing those documents, thus improve the company productivity level.

How we built it

Introduction

PineappleHack is introduced to solve the sustainability report's document processing capabilities. It utilizes the Artificial Intelligent power by leveraging the Azure Cognitive Services under Language Services; for Custom Named Entity Recognition (CNER) and Document Summary, and also Azure Cognitive Search for the semantic graph visualization, to provide those solution.

PineappleHack is designed with features to extract the different custom entity name of the documents by using one of the Cognitive Services features which under Languages services. For this reason, the chosen A.I services are Custom Named Entity Recognition (NER), and the Text Summary features. Language Studio is used to do the process pipeline, as such Upload File, Tagging, Training Data, and Testing Data The scoring result for the training data will be improved time by time.

The Custom Named Entity Recognition (CNER) will train the model to recognize ESG Report structure and content in a custom mode, whereas the Document Summary feature, will summary of the report based on the query and related tag of the sustainable element tag.

The Azure A.I Cognitive Search is used for the Custom Search. Graph Semantic visualization capabilities will also be included in this app for the convenient of the decision maker to find related result of the available sustainability reports of the company. The Power BI will be used to display the visualization feature of the result to the app user.

In addition, this app is designed and built by using Azure Cloud capability which provides security, scalability and cost efficient environment.

Security by using Manage Identity service on top of Azure Active Directory (AAD) service, in Azure cloud environment ,to provide secure key authorization to all the Azure resources and external third party tool to share the A.I capabilities to the client. Scalability is the services offered in future to include different types of sustainability report document and data, to be added and trained by our model. This solution is cost efficient because the Data Lake Gen2 version of our blob container provides better pricing because the data movement in and out of the storage is lesser.

MICROSOFT Responsible A.I and Transparency Notes is applied thoughout the A.I process in this solution.

Responsible A.I for PII is applied for all the tagging document of petronas sustainability reports. Thus all the personal information of the personal and companies are not tagged, thus protect the company confidential, privacy and lawsuit issue.

Challenges we ran into

Difficulty & challenge faced during the design and development process

The difficulty faced during the development period was the configuration setup of my own Azure Cloud environment that has caused the difficulty in performing the AI processing. The configuration of the Azure Active Directory (AAD), Managed Identity and Key Vault need to be synchronized with the Language Studio application, for me to able to process the files in the Language Studio. The process is a bit complex in order to understand and configure the Azure cloud security due to different versions of APIs, and deprecated web services. Another difficulties was I have chosen a wrong approach on solving the problem. I have used a different AI service that does not provides the result I want. However, I managed to tune in this problem into a manageable solution at the last minutes.

After joining this hackathon I feel that I should push myself to join more hackathon in future, for me to advanced my skills in AI. This time is my second time joining CWB hackathon. This time, I pushed myself to take this challenge to enhanced my skill in A.I. However, through the heavy workload and job commitment, i really have time to focus on this hackathon. However, I never give up. I still takes last minutes chance to submit my project, which was 2 days before the deadline. I kept telling myself i don't lose anything because at least i have some result instead of zero, for givng up this challenge. And yess, I improved myself in gaining A.I skills for another 30-50%. And most important thing, i gain my confident to almost 70-80%. Within short time, and pushing myself during my free time during these short time, this is the result of my effort.

Accomplishments that we're proud of

I realized that I gained many skills through the whole project period. My biggest achievement is I NEVER SAY NO till the end! 😊 I can either sleep and enjoy my day, rather then completing this project, but I chose to continue working on it as a positive matter in my A.I journey. For this, I am proud of myself for NEVER GIVE UP!

What we learned

From my hackathon journey experience, I learnt an important lesson in technical perspectives which are:

1. The main references is the always product documentation, Youtubes resource is a no 2 reference. Always keep update with the technical documentation when working in app development.

2. Solving the technical configuration issue of the system is more tougher than solving the AI matter using Azure Cognitive Services. I have underestimate this matter, thus it really effected the development time. If the cloud configuration and the web services is well in place in your computer and laptop, you have less hassle. But somehow solving this issue really gives satisfaction to me and giving me more confident in handling the development. This is the thing that I am afraid, and hate to deal in app development! I hope I will overcome it soon.:)

What's next for Untitled

Go to market (How will the app be available to the public, and is it scalable?) Yes, this app has the capability to improve in AI's market product, in terms of the automated document processing. It also can be scalable to another sustainability reports, by adding the other type of sustainability reports, and other reports that have information about the sustainability information, such as Company Annual Report, etc. The app is possible to be available to the public by having a custom Sustainable Report Recommender.

Built With

• azure
• cognitive
• customnamedentityrecognition(cner)
• documentsummary
• semanticgraphvisualization
• service
• textanalytics