Our inspiration came from the growing need for efficient, scalable AI solutions that can run on modern GPU infrastructure without sacrificing accessibility. We wanted to build something that showcases the power of NVIDIA acceleration combined with Nebius cloud capabilities in a practical, real-world scenario. What our project does is provide an intelligent pipeline that processes and analyzes data in real time, leveraging GPU-optimized models to deliver fast and accurate results through a simple interface. We built it using Python, PyTorch, and NVIDIA CUDA libraries, deployed on Nebius cloud infrastructure with containerized services for reproducibility and scale. The main challenges we ran into were optimizing model inference latency under limited GPU memory and integrating multiple components into a stable end-to-end workflow within the hackathon timeframe. We are proud of achieving near real-time performance on our target workload and of delivering a working demo that clearly demonstrates the value of GPU-accelerated AI in the cloud. Through this project we learned how to profile and tune GPU workloads effectively, how to design resilient cloud-native architectures, and how to collaborate under tight deadlines. What's next for TortTeam is to expand the pipeline to support more data modalities, improve model accuracy with fine-tuning, and explore deployment options that make the solution easier for others to adopt and scale.

Built With

  • cuda
  • docker
  • fastapi
  • github-actions
  • hugging-face-transformers
  • json
  • nebius-ai-cloud
  • nvidia-gpus
  • python
  • pytorch
  • quantization-(int4/int8)
  • react
  • rest
  • speculative-decoding
  • terraform
  • uvicorn
  • vllm
Share this project:

Updates

Submission history