Azure Cosmos DB Developer Cloud Skills Challenge

Screenshot of the custom AI copilot

Inspiration

We believe that education is the key to the development of a prosperous and equitable society. However, we are aware that there are significant challenges in the current education system. Edutech.ao emerges to offer solutions that address these challenges by working in partnership with schools and teachers to provide high-quality content, aligned with national curricula and adapted to the needs of students. We use advanced technologies, such as artificial intelligence and machine learning, to personalize the teaching and learning experience of each student and empower the continuous training of teachers.

What it does

1. Content Search:

Comprehensive Manual Information: Seamlessly access detailed information about the textbooks used by primary school students in the Republic of Angola.
Textbook Overview: Explore the various textbooks available, including their titles, authors, covered content, and other relevant details.
Topic Exploration: Delve into the topics covered in each textbook, gaining insights into learning objectives, related content, and more.
Subtopic Exploration: Uncover the subtopics within each topic, including titles, specific content, suggested activities, and comprehensive information.

2. Personalized Textbook Recommendations:

Tailored Suggestions: Leverage the power of AI to receive personalized textbook recommendations based on your specific questions and learning needs.
Relevant Content Matching: Identify textbooks that align perfectly with the subject matter at hand, ensuring you have the right resources for your learning journey.

3. Natural Language-Powered Content Response:

Seamless Interaction: Engage in natural language conversations with the virtual assistant to seek answers to your questions related to the scientific content of the textbooks.
Clear and Concise Explanations: Receive clear and concise explanations in your own language, fostering deeper understanding and knowledge retention.

Edutech.ao: Your Personalized Learning Companion

How we built it

Data Collection: We initiated data collection by searching for the official Angolan Ministry of Education textbooks for grades 1 to 6, found in PDF format on the government website (www.sepe.gov.ao).
Preprocessing: The PDF files were converted to txt format, removing images and other irrelevant components. The extracted text was processed using a Python algorithm to transform it into structured data.
Organization and Storage: The structured data was organized into an Excel file with the following sheets:
'manuals' (containing title, author, topics, and others)
'topics'
'subtopics' The data was then loaded into a MongoDB database in Azure Cosmos DB, using a Python routine to convert it from Excel into collections in the document database.
Vectorization and Indexes: The text-embedding-ada-002 model was applied to the data to transform it into vectorized fields. Vector search indexes were created to enable semantic similarity search on these fields.
Generative Retrieval Enhancement: LangChain was implemented to improve information retrieval. Based on a prompt, the system performs a vector search on the CosmosDB database to find the most relevant information. The context of the prompt is then passed to the LLM ChatGPT-3.5 model, which generates the response using both the prompt and the context.
Application Development: A backend API application was created using FastAPI to expose the functionality of the LangChain agent. The API was hosted in a Docker container on Azure. Modifications were made to the React frontend application to adapt it to the Angolan educational context.

Challenges we ran into

Data Quality and Standardization: Textbooks may not have consistent formatting or terminology across different grades. This inconsistency can lead to difficulties during data processing and can affect the accuracy of the retrieved information.
Limited Dataset Size: Relying solely on government-issued textbooks might limit the comprehensiveness of the information available. Expanding the dataset with additional educational resources could improve the system's capabilities.
Model Bias and Accuracy: AI models can inherit biases from the data they are trained on. It's important to be mindful of potential bias in the educational materials and to monitor the model's accuracy to ensure it provides fair and representative information.
Computational Resources: Processing large amounts of text data and running complex AI models can require significant computational resources. Optimizing the code and leveraging cloud-based solutions can help address these resource constraints.
Usability and User Interface: Designing a user interface that is intuitive and easy to navigate for educators and students of varying technical backgrounds is crucial.

Accomplishments that we're proud of

Edutech.ao is dedicated to revolutionizing the educational landscape in Angola. The AI-powered virtual assistant is designed to enhance the learning experience for all stakeholders. The platform leverages official school textbooks as its core resource. Natural and intuitive interactions foster deeper engagement with educational content. Edutech plays a pivotal role in democratizing education in Angola.

What we learned

Importance of Data Preprocessing: The project highlighted the importance of carefully cleaning and structuring data before feeding it into AI models. This ensures the accuracy and effectiveness of the information retrieval process.
Benefits of Open-Source Tools: Utilizing a combination of open-source tools like FastAPI and pre-trained models like ChatGPT-3.5 allowed for efficient development and cost-effective implementation.
Potential of Cloud Solutions: Leveraging cloud platforms like Azure Cosmos DB and Docker containers facilitated data storage, scalability, and streamlined deployment.
Need for Ongoing Development: The project serves as a springboard for further development. Future iterations can focus on incorporating additional educational resources, mitigating bias in the AI models, and enhancing the user interface for broader user adoption.
Value of AI in Education: Overall, the project demonstrates the potential of AI technologies to revolutionize education in Angola by providing a more accessible and efficient way to access information and improve learning outcomes.