• 

  Start Processing

• 

  Finished Processing

• 

  Data View in NE:ONE View

Automating HAWB Document Parsing with AI for ONE Record Integration

The Problem

Handling HAWB (House Air Waybill) documents is a time-consuming and error-prone process in logistics operations. These documents are often created manually in PDF format with highly unstructured, formless layouts. Thus, making it difficult to standardize the extraction process and to automate the parsing of critical information. As a result, this manual effort leads to inefficiencies, increased operational costs, and a slower supply chain process.

The ONE Record Challenge We Address

This project addresses Challenge 1: Artificial Intelligence Challenge under the ONE Record initiative. By applying advanced AI techniques, we bridge the gap between unstructured document processing and global logistics standardization.

Our Solution

We have developed an AI-driven solution to extract structured data from unstructured HAWB documents. Using Google Vertex AI’s Large Language Model (LLM), our system is capable of:

1. Parsing HAWB content into structured JSON format, extracting crucial details like shipment data, consignee/shipper data, product descriptions, and item information.
2. Mapping the JSON data into the IATA ONE Record API standard, creating objects such as Products, Items, Pieces, Shipments, Waybills, and Involved Parties.
3. Storing and making structured information accessible via the NE:ONE View, which serves as a platform to easily access and manage logistics objects

The workflow begins when users provides the HAWB documents. Our Golang-based service retrieves the documents, invokes the AI to extract data, and converts the output into ONE Record-compliant structures. This automated process eliminates manual intervention, drastically improving efficiency and accuracy.

How We Built It

• Backend Processing:
  • A Golang microservice fetches the documents and sends them to Google Vertex AI for interpretation.
  • The AI extracts raw data and formats it into a structured JSON that represents all key HAWB fields.
  • These structures are then further translated into the IATA ONE Record API JSON standard and imported to the IATA ONE Record System.
• Sample Data: HAWB documents were utilized to test the AI's ability to accurately extract and map data.

Challenges we ran into

One major challenge was working with an "out-of-the-box" LLM that had no prior training on the specific structure or vocabulary of HAWB documents. This resulted in inconsistencies during parsing. Additionally, some documents lacked mandatory information, requiring us to handle missing fields while still creating compliant outputs for the IATA ONE Record API. These challenges highlighted the importance of refining the model and enhancing data quality.

What We Are Proud Of

• High Accuracy in Data Parsing: The AI-driven data extraction demonstrates exceptional accuracy and adaptability, effectively handling unstructured formats and inconsistent layouts.
• Seamless ONE Record API Integration: Our solution adheres to the industry’s IATA ONE Record standard.
• Efficiency and Scalability: Built with Golang, our backend components deliver high performance and can easily scale to handle huge amount of data.
• Innovation in Logistics Automation: By combining AI and global data standards, we have developed a tool that can transform the way logistics organizations process documents, driving digital transformation across the industry. ## Next Steps To further enhance our solution, we are focusing on:
• Training on Diverse HAWB Documents:
  • Expanding the AI model’s training dataset by incorporating a wide variety of HAWB documents from different regions, logistics operators, and industries.
  • By diversifying our training data, we aim to improve the AI’s adaptability and accuracy in extracting structured data from document formats, like pdf, jpg, etc.
• IoT Integration:
  • Adding IoT capabilities to the system, enabling users to scan HAWB documents directly using their mobile devices or handheld scanners.
  • This would streamline document preprocessing, allowing HAWBs to be uploaded to the system in real-time with minimal effort.
• Developing a User-Friendly API Gateway:
  • Allowing external supply-chain-management-systems to programmatically upload HAWBs and access processed data in real time.

By addressing these next steps, we seek to unlock even greater potential for automating and streamlining global logistics processes.

Built With

• golang
• iata-one-record
• vertexai