DataBridge

LLM Pipeline to automate 90% of supplier communication in oil and gas companies, ensuring accurate procurement data for analysis, saving up to $10 billion in procurement spend per oil and gas company

Comment

Inspiration

In October 2023, I was sponsored by Shell to attend their annual conference in Bangalore as a student innovator. I got an opportunity to interview the CTO of Shell and converse with high-level executives who wanted to leverage AI to improve employee productivity and decision analysis. Coming from a small town in India called Bareilly, this was a huge opportunity for me and I wanted to utilize it to have an impact in the oil and gas industry through my passion for building scalable AI platforms.

Minimizing supply chain risks can reduce 15% of costs, through improving resilience to respond to supply chain disruptions. 30% of Scope 3 emissions can be reduced through supply chain optimization. Scope 3 encompasses emissions that are not produced by the company itself and are not the result of activities from assets owned or controlled by them, but by those that it’s indirectly responsible for up and down its value chain, such as its suppliers. However, to calculate supply chain risks, you need accurate and updated supplier data. Currently, oil and gas procurement employees have to manually upload, verify, and repeatedly ask their suppliers for equipment data, comprising almost 90% of their work time and also increasing the rate of errors: Humans make 100x more data entry errors compared to automated data entry systems. An employee I talked to in the procurement team of a large oil and gas company was manually checking the data accuracy across 10,000 rows and 3,000 needed to be changed.

Major oil and gas companies often employ approximately 3,000 employees solely for the collection and cleaning of supplier data, costing 7 million employee work hours and 76 million USD annually, per company. Moreover, it costs an additional USD 3.25 billion due to the impact of supply chain disruptions which cannot be minimized without accurate supplier data.

I chose to focus on procurement chains because according to Harvard Business Review, purchased products and services account for more than half of the total costs for the average oil and gas company.

In March, I utilized months of market analysis in the supply chain industry to lead a team of passionate high schoolers and design an LLM pipeline that could automate 90% of supplier communication. I was in charge of designing the LLM Pipeline and validating it with experts, which is why I utilized the design for this hackathon in which I am participating solo.

I got over 15 executives' validation in oil and gas companies on how this could be impactful and if we scale the LLM pipeline along with a robust risk analysis, it could save up to 15% of procurement costs, 30% of Scope 3 CO2 emissions, and 90% of procurement employee hours. For a major oil and gas company, it would amount to cost savings of $10 billion in procurement, 46 million tons of Scope 3 CO2 emissions, and 6 million employee work hours, annually.

The LLM pipeline which collects accurate supplier data can be scaled across multiple oil and gas companies. The Global Oil and Gas EPC (Engineering, Procurement, and Construction) Market size was valued at USD 173.29 Billion in 2019 and is poised to grow from USD 182.65 Billion in 2023 to USD 278.19 Billion by 2031, at a CAGR of 5.40 % during the forecast period (2024-2031).

Coming from a financially struggling family, I know how people's employment can change their lives I want to do all that I can to provide businesses with the tools to optimize supply chains because supply chains are the foundation for asset-driven industries that our lives rely on. In the future, I will work on scaling this LLM pipeline to other physical-asset-driven industries such as healthcare and aviation, saving billions of dollars in procurement and millions of employee work hours that are currently being wasted in collecting data.

What it does

The LLM Pipeline I have built in this hackathon can analyze submitted procurement data by the supplier, or purchase agreements from the client to assess any incorrect or inaccurate data by comparing it with JIP33 procurement specifications. Joint Industry Programme 33 (JIP33) procurement specifications are standardized documents that help the oil and gas industry improve the procurement, delivery, and specification of equipment. This saves millions of procurement employee work hours annually, which would otherwise have to check the rows of data with the procurement specifications of each equipment type, across thousands of rows.

After finding the incorrections and inaccurate data, it creates an email draft for the supplier or client to update the specific data with a deadline, creating a forcing function and also helping in tracking supplier communications, saving the back and forth between clients and the suppliers they order from.

How I built it

  1. Collected procurement data: I collected the JIP33 procurement specifications for actuator valves (equipment used in oil and gas assets) and a public purchase agreement of an Indian oil and gas company for Electrohydraulic Actuator for existing Flow Control Valves

  2. Experimented with various prompts and file formats using Google AI Studio

  3. Created a multi-turn conversational chatbot using Vertex AI studio and Gemini API. This chatbot can extract information from text, PDFs, and spreadsheets and analyze them to find corrections and additions. It formats these changes into an email with a deadline to send to the client or supplier depending on which data was being checked.

  4. Created a front-end user interface using Gradio and customized its color palette

Challenges I ran into

  1. Collecting procurement data was challenging since supply chains have a scarcity of large datasets and internal supplier data is proprietary. I utilized public procurement specifications, tender and purchase agreements.

  2. Gradio has a queuing limitation which prevented me from processing large files because of no GPU credits. To solve this, it currently accepts procurement specifications and client/supplier data as text up to a certain length. In the future, I can connect it to GPU credits to process large files seamlessly.

  3. Passing PDFs to the Gemini API required extraction of individual PDF pages, which I understood through experimenting with Google AI studio

  4. Hosting the Gradio app on HuggingFace Spaces was challenging as the local dependencies clashed with each other and caused a lot of bugs. I solved this error through getting the most essential dependencies through pip-chill without versions, so that version dependencies could be resolved.

Accomplishments that I am proud of

  1. Learning how to use Gemini and Gradio solely through their documentation

  2. Hosting my app on HuggingFace Spaces for others to test

What I learned

  1. Building a multi-turn conversational chatbot using Gemini API that can also analyze PDFs

  2. Customizing an app using Gradio

  3. Having an in-depth understanding of procurement specifications

What's next for DataBridge

As this pipeline is scaled in terms of infrastructure, it can refer to contract agreements and procurement specifications to cross-check submitted supplier data. I will also work on integrating it with popular email software so that it can extract data from supplier emails, convert it into a format it can be analyzed in, and also send supplier emails after approval from the employee.

I will add an additional chatbot just for user queries to ensure they have a seamless user experience.

I would also work on tracking supplier communications through the emails sent and received according to the deadline and measuring a supplier performance metric based on the quality and timeliness of the data sent by the supplier, to keep them accountable to submit data timely.

As the model gets trained for longer periods of time, it could also be used to verify the data of older contracts that are still being used in maintenance.

In the attached links,

  1. Gradio app which accepts text for all 3 input fields and returns an email
  2. GitHub repo for longer version of multi-turn conversational chatbot that can accept files
  3. Google Colab Notebook with outputs from the multi-turn conversational chatbot

Built With

  • gemini-api
  • google-ai-studio
  • google-colab
  • gradio
  • python
  • vertex-ai
Share this project:

Updates