Inspiration
From deciphering dolphin clicks to researching how plants communicate distress, we have spent decades attempting to comprehend other species. The majority of methods continue to be species-specific and narrowly focused. We questioned ourselves:
What if we could build a universal semantic bridge between all other living forms and humans using artificial intelligence?
A combination of biosemiotics research, LLM multimodal developments, and a desire to support sustainable farming, wildlife conservation, and interspecies empathy gave rise to the concept. We had the ideal basis thanks to GPT-OSS models, which are open, flexible, and local enough to be adjusted using actual biological data.
What it does
Bidirectional communication between humans and non-human life is made possible by the local agentic system NeuroWeave. It records environmental signals, bioelectrical signals (plant electrophysiology, EEG), and bioacoustic signals (birdsong, whale clicks, and dog barks). Uses a common multimodal semantic embedding space to translate them into natural language spoken by humans. Allows humans to "speak back" in a species' native language by creating reverse translations (e.g., playback of specific whale click patterns). Because it is offline, it can be used in conservation areas, farms, and remote field research. Imagine it as a biology-trained version of Google Translate for life itself.
How we built it
Pipelines for Data Collection:
Audio: Species-specific calls are recorded by microphones and converted into spectrograms and MFCC features. Electrical: Time-series voltage signals are recorded by Arduino/ESP32-based sensors for plant/microbial bioelectric data. Context: Using motion, temperature, and humidity sensors as feature vectors.
Preprocessing and Feature Extraction: Audio → extraction of the Log-Mel spectrogram embedding. Bioelectric → Fourier transforms + spike train analysis. Context → Environmental embeddings are used to encode it.
Multimodal Encoder for Fusion: Trained to align various modalities into a common semantic embedding space using contrastive learning. Locally adjusted with a custom adapter layer and gpt-oss.
Bridge of Languages: From embeddings, GPT-OSS produces text that is readable by humans, and vice versa. A synthesis module tailored to a species replicates electrical patterns, sounds, or signals.
Interaction:
Local Python backend with FastAPI. React-built web-based dashboard for interactive species Q&A.
Challenges we ran into:
Dataset scarcity: We had to merge our own recordings with scholarly datasets because there aren't many multimodal biological datasets. Semantic uncertainty: Since biological signals are probabilistic, the model needs to take "meaning" ambiguity into consideration. Offline optimization is the process of compressing and quantizing gpt-oss models so they can operate locally with minimal accuracy loss. Complexity of reverse translation: Producing precise outputs specific to a species without adding "human bias."
Accomplishments that we're proud of:
The first open-source effort to develop a single multimodal semantic mapping for communication between species. Completely local deployment: it doesn't require the internet to function. Communities can contribute "language packs" for particular species thanks to the modular design. The prototype effectively produced playback calls for basic bird warnings and converted plant drought stress into natural language alerts.
What we learned:
Multimodal alignment is essential because cross-species translation involves context, bioelectricity, and environmental information in addition to audio. Adapter techniques can be used to fine-tune open-source LLMs for biological semantics with relatively small datasets. It takes innovative quantization and inference optimizations to run sophisticated AI offline. When the right tools are used to solve the right problem, the distinction between science fiction and science reality becomes blurry.
What's next for NeuroWeave: The Cross-Species Language Fabric:
Add marine communication, such as warnings about coral bleaching, dolphins, and whales. Connect to wildlife monitoring systems that use drones to provide remote translation in the wild. Provide an open-source toolkit so that scientists can locally adjust species-specific models. Release a study titled "A Unified Semantic Embedding Space for Cross-Species Communication using Open-Source Generative Models." Examine ethical frameworks to ensure non-harmful use of AI-mediated interspecies communication.
Log in or sign up for Devpost to join the conversation.