• 

  Logo

NecroStack: Building an Event-Driven Micro-Framework

Inspiration

The idea for NecroStack emerged from frustration with heavyweight event-driven frameworks. Most solutions like Celery, Kafka consumers, or AWS Lambda orchestrations require significant boilerplate and infrastructure complexity. I wanted something that embodied the Unix philosophy: do one thing well.

The name "NecroStack" plays on the concept of "raising" events—like a necromancer summoning spirits, the framework resurrects messages from queues and breathes life into handlers. The core abstractions follow this theme: Spine (the backbone dispatcher), Organs (the living handlers), and Events (the souls being passed around).

What I Learned

Theoretical Foundations

Building NecroStack deepened my understanding of Event-Carried State Transfer patterns. The key insight is that events should carry all necessary state:

$$E = {id, \tau, type, payload}$$

Where 𝝉 represents the UTC timestamp, ensuring temporal ordering. This eliminates shared mutable state between handlers, giving us:

• Temporal decoupling: Producers and consumers operate independently in time
• Spatial decoupling: No direct references between components
• Synchronization decoupling: Async communication eliminates blocking

Cooperative Multitasking

I learned the nuances of Python's asyncio single-threaded cooperative model. Events process sequentially from a FIFO queue:

$$Q_{t+1} = Q_t - {e_{head}} + {e_1, e_2, ..., e_n}$$

Where each handler can emit, n ≥ 0 new events. This creates elegant chain reactions while maintaining deterministic behavior.

Termination Guarantees

A critical learning was ensuring bounded execution. Without safeguards, recursive event emission creates infinite loops. The solution:

$$\text{steps} \leq \text{max_steps} \implies \text{terminates}$$

If exceeded, the Spine raises RuntimeError, guaranteeing halting for any event chain.

Architecture

The framework follows a circular flow: Backend pulls events → Spine dispatches to matching Organs → Organs handle and emit new events → Backend enqueues emitted events → cycle repeats.

Core Components:

• Event (Pydantic model): Immutable, validated, JSON-serializable messages with UUID v4 IDs and UTC timestamps

• Organ (Abstract base class): Handlers declare subscriptions via listens_to and implement sync or async handle() methods

• Spine (Dispatcher): Routes events to matching Organs, manages failure modes, tracks metrics

• Backends: Pluggable storage—InMemoryBackend for testing, RedisBackend for production with consumer groups and DLQ support

Technology Stack: Python 3.11+ with native asyncio, Pydantic v2 for validation, Redis Streams for durable messaging, pytest + Hypothesis for property-based testing.

Challenges Faced

1. TOCTOU Race Conditions

The initial bounded queue implementation had a time-of-check-to-time-of-use race between checking if the queue was full and actually inserting. The fix was using atomic put_nowait() operations that raise QueueFull immediately.

2. Handler Timeout Management

Async handlers could hang indefinitely. Implementing timeouts required detecting coroutines via inspect.iscoroutine() and wrapping with asyncio.wait_for().

3. Redis Consumer Group Complexity

Implementing at-least-once delivery with Redis Streams involved automatic consumer group creation, pending message recovery via XPENDING/XCLAIM, dead-letter queues for poison messages, and connection pooling with automatic reconnection. Mapping event UUIDs to Redis stream IDs for acknowledgment was particularly tricky.

4. Memory Leak Prevention

Retry tracking in handlers initially used unbounded dictionaries. Events that failed permanently would accumulate entries forever. The fix used TTL-backed caches with cachetools.TTLCache.

5. Silent Failure Suppression

Early versions had except: pass blocks that swallowed ack failures silently. This violated observability principles—failures must be visible and tracked in metrics.

Results

NecroStack achieves its design goals:

• 140 tests passing with property-based testing via Hypothesis
• Zero external dependencies for core functionality
• Production-ready Redis backend with health checks and metrics
• Structured JSON logging with correlation IDs throughout

The framework proves that event-driven architectures don't need to be complex. Sometimes, three well-designed abstractions are all you need.

Built With

• asyncio
• cachetools
• github
• hypothesis
• kiro
• pydantic
• pytest
• python
• redis
• ruff