BioStream Multimodel AI

Turning raw lab screenshots and data log streams into structured clinical summaries instantly."

Comment

Inspiration

In biological research labs, biotech firms, and academic classrooms, interpreting raw data from cell culture tracking logs represents a major operational bottleneck. Researchers constantly capture visual screenshots of logging streams, spreadsheets, and system metrics. Manually transcribing these values, checking morphology statuses, and writing health summaries introduces human error and slows down discovery timelines. We wanted to build an autonomous agent that takes these messy logs and screenshots and turns them into a standardized structural layout instantly.

What it does

BioStream Multimodal AI acts as a secure, intelligent parsing bridge for automated cell culture monitoring. The application accepts direct user uploads of data logs and visual dashboard screenshots. When a user feeds a screenshot log stream into the terminal—using our primary validation string: "Analyze this lab image. Data log stream: cell_conf: 0.78, active_mitosis_count: 14, morphology_status: stable, stress_ind: null." The agent immediately activates its internal chain-of-thought loop. It strips away formatting noise, identifies key cell parameters, evaluates proliferation rates, and dynamically outputs a deterministic, clinical-grade overview report detailing confluence density, active division nodes, and sample viability.

How we built it

The application is constructed completely without traditional code by utilizing a high-performance visual orchestration pipeline on Dify Cloud. • The Ingestion Layer: We enabled native visual file and snapshot upload nodes to accept direct user screenshots alongside natural language text strings. • The Processing Engine: The data pipeline is routed directly to an advanced gpt-oss-120b large language model architecture hosted via GroqCloud infrastructure to guarantee lightning-fast inference and zero processing lag. • Prompt Engineering Integration: We designed a constrained system instruction matrix that forces the underlying reasoning engine to parse variables deterministically, bypass environmental noise, and structure outputs using strict Markdown schemas.

Challenges we ran into

  • Model Deprecations: Managing backend server route changes on Groq caused unexpected status code 404 and 400 communication errors when initial vision test models were decommissioned mid-development.
  • Input Modality Constraints: Adapting a text-heavy model architecture to handle visual data assets required a strategic structural pivot from direct image matrix ingestion to high-fidelity screenshot data-stream parsing.
  • Format Enforcement: Constraining an advanced open-weights reasoning model to completely drop conversational chatter and output purely deterministic, predictable Markdown schemas without manual rule-writing.

Accomplishments that we're proud of

We were able to simplify images of complex cells

What we learned

  • Visual Orchestration: Discovered the speed and efficiency of building scalable, end-to-end AI agent pipelines using no-code visual frameworks on Dify Cloud.
  • Prompt Engineering Boundaries: Learned how to embed strict system guardrails and conditional fallback directives to guide text models when processing unstructured text data inputs.
  • Open-Weights Infrastructure: Gained deep insights into token execution speeds, server latencies, and how to successfully route data parameters through the high-performance gpt-oss-120b model framework on Groq.

What's next for BioStream Multimodel AI

Moving forward, we plan to directly expand our visual node pipelines by connecting the agent's markdown parser to open-source medical lookup engines (like PubMed) to let the AI agent autonomously verify and cite peer-reviewed scientific journals for every visual anomaly or cell parameter it isolates.

Built With

  • agent
  • ai
  • dify
  • engineering
  • groqcloud
  • llm
  • no-code
  • prompt
Share this project:

Updates

Submission history