CiliTest

Cilium Network Policy Dashboard: A comprehensive tool for converting, validating, and visualizing Cilium Network Policies with a clean web-based dashboard.

Comment

Inspiration

Managing network policies in Kubernetes environments can be overwhelming, especially when dealing with Network Policies. We noticed that many developers and DevOps engineers struggle with:

  • Converting legacy firewall rules to modern CNI policies
  • Validating complex YAML configurations
  • Understanding the actual network topology their policies create
  • Debugging policy misconfigurations that lead to connectivity issues

This inspired us to create CiliTest - a comprehensive tool that makes Cilium Network Policy management accessible, visual, and error-free. We wanted to bridge the gap between traditional firewall rules and cloud-native networking by providing an intuitive interface that both beginners and experts could use.

What it does

CiliTest is a full-featured Cilium Network Policy management platform that combines three critical capabilities:

1. Intelligent Conversion

  • Transforms JSON firewall rules into properly structured Cilium Network Policy YAML
  • Automatically groups rules by source endpoint for optimal policy organization
  • Preserves port protocols and destination mappings accurately

2. Comprehensive Validation

  • Performs multi-layer validation: YAML syntax, schema compliance, and style linting
  • Validates against the official Cilium Network Policy schema
  • Provides actionable error messages with line numbers and suggestions
  • Detects common misconfigurations before deployment

3. Interactive Visualization

  • Generates network topology graphs showing service-to-service connections
  • Displays policy analytics with port usage and connection statistics
  • Offers both web dashboard and CLI interfaces for different workflows
  • Creates exportable visualizations for documentation and presentations

How we built it

Technology Stack

  • Frontend: Streamlit for the web dashboard with a clean, professional UI
  • Visualization: Plotly for interactive graphs and NetworkX for topology analysis
  • Validation: Custom validation engine using yamllint, jsonschema, and Python YAML parsers
  • CLI: Typer for command-line interface with Rich for beautiful terminal output
  • Testing: Mock and real Kubernetes connectivity testing capabilities

Architecture

We designed CiliTest with modularity in mind:

  • converter.py: Handles JSON-to-YAML transformation with intelligent grouping
  • validator.py: Multi-stage validation pipeline with detailed reporting
  • visualizer.py: Network graph generation with matplotlib and ASCII fallbacks
  • tester.py: Policy testing with both mock and real cluster connectivity checks
  • cli.py: Comprehensive CLI with workflow automation
  • dashboard.py: Streamlit web application with session management

Development Process

  1. Started with core conversion logic to handle firewall rule transformation
  2. Built the validation engine with progressive checks (syntax → schema → style)
  3. Added visualization capabilities with both graphical and terminal outputs
  4. Created the web dashboard for accessibility
  5. Implemented CLI for automation and CI/CD integration
  6. Refined UX based on real-world usage patterns

Challenges we ran into

1. Complex Schema Validation

Cilium Network Policy has a deeply nested schema with many optional fields and combinations. Creating a validator that accurately checks all edge cases while providing helpful error messages was challenging. We solved this by implementing a multi-stage validation pipeline and leveraging JSON Schema's Draft7Validator.

2. Graph Visualization Performance

Rendering complex network topologies with many services became slow and cluttered. We optimized by:

  • Implementing spring layout algorithms with tuned parameters
  • Adding intelligent node sizing based on connectivity
  • Providing ASCII fallback for terminal-only environments
  • Caching graph layouts for repeated visualizations

3. Session State Management

Managing policy state across Streamlit page navigation was tricky since Streamlit reruns the entire script on each interaction. We implemented careful session state management to persist policies, validation results, and user preferences across page changes.

4. Cross-Platform Compatibility

Supporting both Windows PowerShell and Unix shells required careful handling of:

  • Path separators and file operations
  • Terminal encoding (especially for UTF-8 characters)
  • Command execution differences
  • Virtual environment activation scripts

5. Real vs. Mock Testing

Integrating real Kubernetes cluster testing while providing useful mock tests for users without clusters required building a flexible testing framework that could gracefully fallback and provide meaningful results in both scenarios.

Accomplishments that we're proud of

User-Friendly Interface: Created a clean, accessible web dashboard that makes complex policy management approachable for all skill levels.

Comprehensive Validation: Built a validation engine that catches errors before deployment, potentially preventing production incidents.

Visual Network Understanding: Transformed abstract YAML policies into clear, interactive network topology diagrams that make understanding policy impact immediate.

Dual Interface: Provided both GUI and CLI interfaces, supporting interactive exploration and automated workflows.

Rich Analytics: Implemented policy statistics and metrics that help users understand policy complexity and coverage.

Production-Ready Code: Clean, modular codebase with proper error handling, type hints, and comprehensive documentation.

Zero-to-Running in Minutes: Simple installation process with clear documentation gets users productive quickly.

What we learned

Technical Learnings

  • Schema Design: Deep understanding of Cilium Network Policy schema and Kubernetes resource specifications
  • Graph Theory: Practical application of network topology algorithms for visualization
  • Streamlit Architecture: Advanced patterns for state management and multi-page applications
  • Validation Patterns: Building multi-stage validation pipelines with meaningful error reporting

Process Learnings

  • User-Centric Design: Importance of clear error messages and visual feedback in developer tools
  • Progressive Enhancement: Value of providing both simple defaults and advanced options
  • Documentation First: Good documentation reduces support burden and improves adoption
  • Modularity Pays Off: Well-separated concerns made adding features and fixing bugs much easier

Domain Knowledge

  • Network Policies: Deep dive into Cilium CNI, eBPF-based networking, and Kubernetes network security
  • DevOps Workflows: Understanding how network policies fit into CI/CD pipelines and GitOps practices

What's next for CiliTest

  • Cloud Service: Hosted version for teams without local setup
  • Collaboration Features: Multi-user policy editing and review workflows
  • Security Scanning: Identify overly permissive policies and security risks
  • Policy Library: Community-contributed policy patterns and best practices
  • Real-time Monitoring: Integration with live clusters for policy impact analysis
  • Learning Platform: Interactive tutorials for learning Cilium network policies

Built With

Share this project:

Updates