OnCloud

🧩 Project: OnCloud — Secure, Scalable & Intelligent AWS Pipeline

📌 Overview

In this cloudathon, we designed and implemented an end-to-end cloud solution on AWS that focuses on data security, automation, observability, and AI-driven insights. The system integrates multiple AWS services to handle real-world challenges in modern cloud environments.

🔐 Data Security & Encryption at Scale

We built a secure data lake pipeline by:

Encrypting large-scale S3 data using SSE-KMS (Customer Managed Keys)
Using S3 Batch Operations for scalable re-encryption of existing objects
Troubleshooting manifest and permission issues (KMS, IAM roles, inventory formats)
Enforcing default bucket encryption for all future uploads

👉 Outcome: Achieved secure, policy-compliant encryption across all data at rest.

⚙️ Serverless Audit Logging Pipeline

We developed an automated monitoring pipeline using:

Amazon RDS (MariaDB Audit Plugin) to generate audit logs
CloudWatch Logs for centralized log collection
AWS Lambda to export logs programmatically
Amazon S3 as durable storage
EventBridge for scheduled execution (daily automation)

👉 Features:

Handles concurrent export conflicts
Implements retry logic
Uses environment-driven configuration

👉 Outcome: Fully automated, serverless audit log export system ensuring compliance and observability.

🧠 AI-Powered Data Analysis

We integrated:

Amazon Bedrock (Nova Lite)
Amazon SageMaker

to:

Extract financial data from documents (PDF/CSV)
Perform reconciliation and calculations
Generate business insights using prompt engineering

👉 Outcome: Demonstrated how AI can enhance data analysis workflows in cloud systems.

🏗️ Infrastructure Debugging & Optimization

We solved real-world DevOps issues:

Fixed CloudFormation template errors (GetAtt, runtime deprecations, security rules)
Migrated databases to Graviton (ARM64) for better performance
Handled IAM least-privilege constraints using predefined roles

👉 Outcome: Improved infrastructure reliability, performance, and security.

🚀 Key Achievements

Completed 10+ cloud challenges
Used 6+ AWS services
Built a production-style cloud architecture
Ranked on leaderboard under time constraints

💡 Key Learnings

S3 Batch Operations is essential for large-scale data changes
Serverless pipelines enable cost-efficient automation
IAM and KMS are critical for secure cloud design
AI services can be integrated for intelligent data processing

🎯 Conclusion

This project showcases how to build a secure, scalable, and intelligent cloud ecosystem by combining automation, security best practices, and AI capabilities using AWS.