Cortex: Cognitive Perception for Elasticsearch Agents

Inspiration

Most AI agents dump every sensor event into Elasticsearch, creating noisy indices that degrade search quality and waste resources. Meanwhile, human brains filter ~99% of incoming signals through habituation, circadian rhythms, and salience detection before anything reaches conscious reasoning.

We asked: what if your Elasticsearch agent had the same cognitive filter? What if it could decide what is worth indexing and reasoning about, before a single document hits the cluster?

What it does

Cortex is a cognitive-science-based perception framework that sits between sensors and Elasticsearch. It applies three neuroscience mechanisms before any indexing or agent action:

1. Habituation Filter — repeated stimuli raise the threshold (you stop noticing the clock ticking). Based on Thompson & Spencer (1966).
2. Circadian Rhythm — adjusts vigilance by time of day (night mode heightens alertness). Based on Borbely (1982).
3. Decision Engine — routes events by priority using a salience network inspired by Corbetta & Shulman (2002).

The Elasticsearch bridge (cortex/bridges/elasticsearch.py) provides cognitive-filtered indexing: events pass through Cortex's perception pipeline first, and only novel, significant events get indexed into Elasticsearch with enriched metadata (cognitive scores, priority levels, timestamps). This means your ES indices contain only what matters, dramatically improving search relevance and agent reasoning quality.

The framework also includes an MCP Server with 11 tools for multi-step agent workflows: querying perception state, adjusting filter sensitivity, reviewing notification queues, and triggering actions based on cognitive assessment.

Real-world validation: In a 48-hour test with 2,200+ sensor events from live camera feeds, Cortex filtered 91% of noise — reducing 1,980 motion events to 173 conscious-level alerts, with 170+ orienting responses correctly identified.

How we built it

• 7,169 lines of Python across 7 cognitive modules, 4 sensor sources, 2 integration bridges (Elasticsearch + Gemini), and an MCP Server — 61 commits
• Pure Python, zero dependencies — stdlib-only core, Elasticsearch client only needed for the ES bridge
• Config injection pattern — one CortexConfig object wires everything
• Elasticsearch bridge with cognitive metadata enrichment: each indexed document includes habituation state, circadian phase, priority score, and orienting response markers
• MCP Server exposing 11 tools for agent orchestration (JSON-RPC subprocess protocol)
• 188 tests running in under 4 seconds, including end-to-end MCP tool verification
• CLI entry points: cortex-serve (MCP server) and cortex-replay (real-world demo)

Architecture:

Sensors → Cortex (perception) → Elasticsearch (knowledge)
          habituation filter     cognitive-filtered indexing
          circadian rhythm       enriched metadata
          priority assessment    search-ready documents
                ↓
          MCP Server (11 tools) → LLM Agent → Actions
          perception state         multi-step reasoning
          filter controls          ES|QL queries
          notification queue       alert / investigate / log

Challenges we ran into

• Balancing filter sensitivity for indexing: Too aggressive filtering means missing events that should be searchable later; too permissive defeats the purpose of cognitive filtering. We tuned thresholds using cognitive science literature and validated against 48 hours of real-world data.
• Metadata schema design: Designing Elasticsearch document mappings that capture cognitive state (habituation level, circadian phase, priority) while remaining queryable with ES|QL required careful iteration.
• MCP tool design: Exposing the right granularity of cognitive controls — too few tools limits agent capability, too many creates decision paralysis. We settled on 11 tools covering perception, filtering, and action.

Accomplishments that we're proud of

• 91% cognitive load reduction validated on 48+ hours / 2,200+ real-world sensor events
• 7,169 lines of Python, 61 commits — production-ready framework with comprehensive test coverage
• 188 tests passing in 3.42 seconds — well-tested, reliable
• Zero external dependencies for core — stdlib Python, works anywhere 3.10+ runs
• 11 MCP tools for multi-step agent workflows, all E2E verified
• Cognitive-enriched ES indexing — every document carries perception metadata for smarter queries
• Real-world validated with live camera motion detection, YOLO classification, and circadian-aware alerting

What we learned

• Cognitive science principles translate directly to practical Elasticsearch optimization — habituation alone reduces index size by 60-80% while improving search relevance
• Pre-filtering dramatically improves agent reasoning quality — LLMs give better answers when Elasticsearch contains only novel, important events
• The MCP protocol is excellent for exposing cognitive state to LLM agents — tools like get_perception_state and adjust_filter_sensitivity give agents meta-cognitive control

What's next for Cortex

• ES|QL cognitive queries — custom query templates that leverage cognitive metadata (e.g., "show me all orienting responses in the last hour" or "events above habituation threshold during night mode")
• Elastic Agent integration — deploy Cortex as an Elastic Agent input plugin for production environments
• ReachyMini robot integration — physical robot body with camera, microphone, and IMU sensors feeding through Cortex into Elasticsearch for embodied AI agent workflows
• Streaming perception — real-time continuous filtering with Elasticsearch ingest pipelines
• PyPI publication — pip install cortex-agent for easy adoption

Built With

• agent
• ai-agents
• borbely-1982
• borbely1982
• circadian-rhythm
• cognitive-science
• cognitive-science-(thompson-&-spencer-1966
• cognitivescience(thompson&spencer1966)
• corbetta&shulman2002
• elasticsearch
• habituation
• mcp
• mcp-(model-context-protocol)
• perception
• python
• zero-dependencies