OCDS Smart Search

Unlock transparency in government data with OCDS Insight Hub: an AI-powered platform making public contracting data accessible, searchable, and actionable for citizens, journalists, and officials.

Comment

Inspiration

I have always been a person who loves learning and improving the things around me. Technology has been a big passion of mine, and I’ve always tried to use my knowledge to help in my country, Guatemala, by optimizing processes that might not be done as well as they could be, especially when those improvements can help society and government.

In Guatemala, all government entities use very outdated technology, old programming languages, and mostly on-premises infrastructure. The performance of government websites is among the worst, which significantly limits transparency in our country. For data enthusiasts or journalists, creating investigative reports or even performing research on corruption can be extremely challenging, as these sites lack the necessary tools to provide accessible information. The only good decision they’ve made is adopting the OCDS standard for all government transactions.

The Open Contracting Data Standard (OCDS) is an open data standard used worldwide for improving transparency and accountability in public procurement. Supported by the G7, G20, and OECD, OCDS is widely implemented by governments, organizations, and institutions dedicated to creating open, accessible data. This standard structures public contracting data in a consistent format, making it easier to understand and analyze. Within OCDS, buyers—typically government entities—publish information about tenders, while bidders—organizations or individuals—participate in these tenders by submitting bids. The structured format helps stakeholders follow the entire process from bidding to contract award and completion, promoting transparency and facilitating analysis for all users.

However, in Guatemala, access to these records is provided through Excel files with updated data. This makes it very difficult to analyze metrics, detect anomalies, or observe trends within each government entity. What’s sad is that having a stable and fast platform isn't even financially prohibitive, especially with today’s tools like cloud computing and databases capable of handling millions of records without issues, like MongoDB. Additionally, there’s no solution that works efficiently with OCDS to offer powerful semantic searches for quickly finding contracts or purchases by keyword—a gap that exists both in Guatemala and other countries using OCDS.

Given all this, I decided to implement a solution to improve the way OCDS information is displayed and accessed. My goal is to provide citizens, reporters, researchers, or activists with a simple way to obtain important metrics on government entities.

What it does

Our platform offers a seamless way to access OCDS data based on buyers, bidders, date ranges, and semantic searches powered by MongoDB’s vector-based indexing. This makes it easier for users to find relevant information quickly and accurately. Additionally, we implemented a user agent using AWS Bedrock to create a chatbot that can deliver information on any OCDS record, as well as provide insights on buyers and bidders.

With this solution, users can retrieve critical metrics, identify spending trends, detect anomalies, and access records faster than ever. It empowers citizens, journalists, and officials to engage with and understand public spending transparently and efficiently.

How we built it

For this hackathon, we focused on building a robust platform using MongoDB’s vector-based indexing for semantic searches. First, we collected OCDS data and structured it to enable effective filtering and retrieval. We then built a search feature that allows users to query data by buyer, bidder, and date range, with the added capability of performing semantic searches on specific terms, like product names, locations, and categories.

To further enhance the user experience, we developed a chatbot using AWS Bedrock. This chatbot interacts with our platform’s backend, responding to queries about specific buyers, bidders, and general OCDS metrics. For the chatbot’s frontend, we used a React library compatible with Ant Design, streamlining the chatbot’s appearance and functionalities.

Challenges we ran into

One of the main challenges was obtaining the data itself. In Guatemala, there isn’t much documentation on OCDS processes, and the files published with new OCDS data are not incremental. I had to manually identify changes in each OCDS file and input them into the database, making the initial data collection process challenging.

Once the data was available, the next challenge was deciding how to create embeddings for storing vectors for our vector search index. Ultimately, an embedding that stores all tender details, like bidder information, required products or services, quantities, and locations, proved sufficient for now. With this data embedded, we created a vector index, and the results were impressive. Users could search for specific terms and immediately find all tenders related to that search.

Though it wasn’t necessary for this hackathon, we also implemented Atlas Search features, like autocomplete, which was helpful for filtering through all the buyers and bidders in the country whenever a user wanted to search by those parameters. Implementing the chatbot was also challenging, as we needed to ensure that the prompts and logic in AWS Bedrock interacted correctly with our platform’s backend.

Accomplishments that we're proud of

We’re proud to have built a powerful solution that combines advanced technologies to make government data more accessible and transparent. Our platform performs seamless semantic searches and offers an intuitive chatbot experience, making complex data easily accessible to anyone. Integrating these tools in a way that optimizes both speed and usability was a rewarding achievement, and we’re thrilled with the positive impact this solution can have on transparency and accountability in public data.

What we learned

Through this project, we learned the importance of efficient data structuring and indexing to make large datasets accessible. We also deepened our skills in prompt engineering for effective chatbot interactions. Working with AWS Bedrock and MongoDB’s vector capabilities gave us insight into the latest technologies and how they can transform public data accessibility. This experience reinforced the value of accessible government data for fostering transparency and empowering citizens.

What's next for OCDS Smart Search

The possibilities for OCDS with this integration are limitless. This system can enable the detection of corruption cases, reveal hidden spending trends, and even identify irregularities, bringing a level of transparency that could genuinely benefit the country and foster greater trust in public institutions. In the future, we plan to enhance our platform by integrating more advanced AI capabilities, such as predictive analytics, to uncover trends and anomalies proactively. Additionally, we want to expand this solution to other countries using OCDS, helping them make their data more accessible and transparent.

Built With

Share this project:

Updates