ReturnShield AI

• 

  broken product files for sample execution

• 

  broken product files for sample execution

Project Story: Return Integrity with Amazon Nova (Hackathon Submission)

Inspiration
I was inspired by real-world retail return abuse and the need for inclusive AI that can detect sophisticated fraud while helping human operators. I wanted a system that runs combined text+vision reasoning, and also supports accessibility (voice output for blind users).

What I built

• Multi-agent pipeline (Intake, Evidence, Behavior, Policy, Action, Audit)
• Amazon Nova 2 Lite for reasoning and multimodal analysis
• Nova 2 Sonic text-to-speech for audio outputs
• Streamlit dashboard with:
  • case scoring
  • detailed evidence tabs
  • 🔊 Read Aloud buttons per section
• FastAPI endpoint for API integration
• Docker + docs + deployment guides

What I learned

• How to orchestrate agent-based workflows in Python
• How to handle model calls and fallbacks with AWS Bedrock
• Issues with GitHub 100MB file limits (historical file removal)
• 6-agent results conversion and Streamlit session state
• Real LRS vs offline caching for multimedia data

How I built it

1. Setup environment: bash python -m venv .venv .\.venv\Scripts\Activate.ps1 pip install -r requirements.txt
2. Add .env:
   • AWS_REGION=us-east-1
   • AWS_ACCESS_KEY_ID=...
   • AWS_SECRET_ACCESS_KEY=...
   • NOVA_SONIC_MODEL_ID=us.amazon.nova-2-sonic-v1:0
3. Run tests: bash python -m src.main
4. Generate sample cases: bash python generate_results.py
5. Start dashboard: bash streamlit run dashboard/app.py
6. (Optional API) bash uvicorn api.server:app --reload

Challenges faced

• Large file limit: Return_shield_AI.mp4 >100MB caused push failure; required history cleanup with git filter-repo / BFG.
• Session-state bugs: ground_truth UnboundLocalError, fixed by immediate initialization and safe set in st.session_state.
• Nova API auth: InvalidSignatureException / polly:SynthesizeSpeech AccessDenied; resolved by correcting AWS key and policy.
• Nova Sonic model availability: some regions not yet supported; used fallback to AWS Polly and code path.

Performance metrics

• Precision ~75%, Recall ~100%, F1 ~85.7%
• Avg latency ~2.5s per case
• Action distribution: manual review vs enhanced verification

Code reference links

• README
• nova_client.py
• orchestrator.py
• app.py
• server.py
• scenarios.py

!Architecture

Math / model scoring

In risk scoring we used weighted sum: $$ \text{risk} = w_E E + w_B B + w_P P $$ with normalization: \(E, B, P \in [0,1]\), and final level thresholds:

• low: \(<0.25\)
• medium: \(0.25 \le \text{risk} < 0.6\)
• high: \( \ge 0.6 \)

Full Helm: pipeline decisions are ultimately based on:

score = 0.4 * evidence_score + 0.35 * behavior_score + 0.25 * policy_score

Final note: this project is built to demonstrate end-to-end fraud prevention using Amazon Nova and is ready for hackathon review.

Built With

• agents
• amazon-web-services
• boto3
• fastapi
• nova
• numpy
• pandas
• python
• pyyaml
• streamlit
• vscode