Tempus
A Real-Time Veterinary Microscopy Copilot
Tempus transforms a Raspberry Pi 5 running QNX into an intelligent microscopy workstation.
It captures live specimens, detects blood cells, measures visible structures, and gives veterinary laboratory technicians a dedicated interface for reviewing and correcting results.
Those human-verified corrections create a feedback loop that can continuously improve future versions of the model.
Tempus does not just analyze specimens—it learns from the people who understand them.
Inspiration
Veterinary microscopy remains highly manual.
Laboratory technicians inspect specimens, count cells, document observations, and transfer their findings between disconnected systems. This takes time, introduces inconsistency, and makes collaboration difficult.
We asked:
Could an affordable, real-time microscope assistant accelerate this workflow while still keeping the laboratory technician in control?
That question became Tempus.
What It Does
Tempus can:
- Stream live microscope imagery
- Detect red blood cells, white blood cells, and platelets
- Draw real-time detection overlays
- Count visible cellular structures
- Measure cell area and specimen coverage
- Describe basic morphology
- Generate AI-assisted observations
- Let technicians confirm, reject, or correct detections
- Save verified annotations for future model training
- Preserve findings for reports and historical comparison
Tempus is a decision-support and documentation platform—not an autonomous diagnostic system.
System Architecture
flowchart LR
A["Specimen"] --> B["Microscope"]
B --> C["Camera Module 3"]
C --> D["Raspberry Pi 5 + QNX"]
D --> E["Native C Frame Capture"]
E --> F["Secure SSH Tunnel"]
F --> G["FastAPI Backend"]
G --> H["YOLO Blood-Cell Model"]
H --> I["OpenCV Measurements"]
I --> J["Tempus Dashboard"]
J --> K["Lab Technician Review"]
K --> L["Verified Annotation Dataset"]
L --> M["Model Retraining"]
M --> H
D --> N["ST7789 Display"]
The Raspberry Pi and QNX handle deterministic camera acquisition and hardware control.
The connected workstation currently runs the YOLO model, OpenCV analysis, API, and Tempus dashboard.
A human-in-the-loop feedback system allows laboratory technicians to verify the model’s output before corrections are included in a future training dataset.
Data Flow
sequenceDiagram
participant Camera
participant QNX
participant API
participant Model
participant Technician
participant Dataset
Camera->>QNX: Capture NV12 frame
QNX->>API: Stream frame securely
API->>Model: Run blood-cell detection
Model-->>API: Classes, boxes and confidence
API-->>Technician: Display annotated specimen
Technician->>API: Confirm, reject or correct
API->>Dataset: Store verified annotation
Dataset-->>Model: Retrain future model version
- QNX captures a raw NV12 camera frame.
- A custom C bridge packages the frame and its metadata.
- The frame travels through an authenticated SSH tunnel.
- FastAPI converts it into an OpenCV image.
- YOLO detects visible blood cells.
- OpenCV calculates measurements and morphology features.
- Tempus displays the results to the technician.
- The technician confirms, rejects, or corrects each result.
- Verified corrections are saved as training annotations.
- Curated annotations improve future model versions.
Human Feedback Loop
The laboratory technician remains the final authority.
flowchart TD
A["AI Detection"] --> B["Technician Review Page"]
B --> C["Confirm Detection"]
B --> D["Correct Class"]
B --> E["Adjust Bounding Box"]
B --> F["Reject Detection"]
B --> G["Add Missed Cell"]
C --> H["Verified Annotation"]
D --> H
E --> H
F --> H
G --> H
H --> I["Quality Review"]
I --> J["Curated Training Dataset"]
J --> K["Retrain and Validate"]
K --> L["Versioned Model Release"]
L --> A
The review page is designed to let a technician:
- Confirm a correct detection
- Change an incorrect cell class
- Adjust a bounding box
- Remove a false detection
- Add a cell the model missed
- Attach a note to an unusual structure
- Mark the specimen as unsuitable for training
Corrections are not immediately learned from blindly.
They first enter a curated dataset where they can be reviewed, anonymized, versioned, and validated before retraining. This prevents a single mistake from silently changing the production model.
Why the Feedback Loop Matters
Blood-cell appearance varies across:
- Species
- Staining methods
- Microscope optics
- Camera settings
- Lighting conditions
- Specimen quality
- Clinical environments
A model trained on one dataset may not generalize perfectly to every veterinary clinic.
Tempus turns normal laboratory review into a structured learning opportunity. Over time, verified corrections can help the system become more representative of real veterinary specimens and local laboratory conditions.
The AI assists the technician, and the technician improves the AI.
Native QNX Camera Pipeline
Tempus uses QNX 8.0 on a Raspberry Pi 5 to control the camera and embedded hardware.
The QNX layer handles:
- Sony IMX708 sensor initialization
- 2304×1296 NV12 frame capture
- Camera buffers and timestamps
- Y and UV image planes
- GPIO control
- SPI communication
- Experimental local display output
Python camera libraries were unavailable on the QNX image, so the acquisition system was implemented using QNX’s native C camera API.
Secure Frame Transport
The camera produces raw NV12 data rather than browser-ready images.
We created a binary protocol containing:
- Frame identifier
- Width and height
- Y-plane stride
- UV-plane offset
- UV-plane stride
- Pixel format
- Payload size
- Raw image data
Frames travel through an authenticated SSH tunnel rather than an exposed camera service.
Blood-Cell Detection
Tempus uses a custom YOLO model to identify:
RBCWBCPlatelet
The model produces:
- Bounding boxes
- Cell classifications
- Confidence scores
- Total cell counts
- Counts by class
OpenCV provides additional measurements, including:
- Visible area
- Specimen coverage
- Circularity
- Aspect ratio
- Solidity
- Contour irregularity
These measurements provide descriptive visual information without claiming medical diagnosis.
Technician Experience
flowchart LR
A["Live Capture"] --> B["Run Detection"]
B --> C["Review Results"]
C --> D["Correct Mistakes"]
D --> E["Approve Specimen"]
E --> F["Generate Report"]
E --> G["Submit Training Feedback"]
The Tempus dashboard provides:
- Live microscope video
- Detection overlays
- RBC, WBC, and platelet counts
- Confidence controls
- Area and coverage measurements
- Morphology summaries
- Capture and report controls
The technician review page adds:
- Detection confirmation
- Class correction
- Bounding-box editing
- False-positive removal
- Missed-cell annotation
- Review notes
- Training-data consent controls
Embedded Display
Tempus also includes an experimental display path for a two-inch 240×320 ST7789 screen.
The QNX display driver uses:
- SPI communication
- GPIO-controlled reset
- Data/command signaling
- Backlight control
- NV12-to-RGB565 conversion
- Real-time image downscaling
flowchart LR
A["QNX Camera Frame"] --> B["NV12 Conversion"]
B --> C["320×240 Downscale"]
C --> D["RGB565"]
D --> E["SPI"]
E --> F["ST7789 Display"]
Built With
QNX 8.0, Raspberry Pi 5, Raspberry Pi Camera Module 3, Sony IMX708, QNX Sensor Framework, C, Python, FastAPI, WebSockets, OpenCV, Ultralytics YOLO, NumPy, React, Vite, SSH tunneling, SPI, GPIO, ST7789 display
The Original Microscope Camera Challenge
Our original AmScope MU1000 camera used a proprietary ToupTek USB protocol instead of standard UVC.
We successfully:
- Identified its USB endpoints
- Queried its firmware and hardware versions
- Read its calibration EEPROM
- Reproduced its vendor handshake
- Received the expected
0x08acknowledgement
Full streaming still required an undocumented model-specific sensor initialization sequence.
We completed the prototype using the officially supported Camera Module 3 while preserving the ToupTek work for a future native QNX driver.
Challenges
QNX Camera Integration
Python camera libraries were unavailable, so we built a native C acquisition bridge using the QNX camera API.
Raw Image Reconstruction
The camera produced NV12 buffers with platform-specific offsets and strides. Tempus reconstructs each frame using the metadata supplied by QNX.
Secure Networking
Direct access to the custom frame port was unavailable. We transported frames through an authenticated SSH tunnel.
Real-Time AI
Camera acquisition and AI inference have different performance requirements. Separating them keeps QNX focused on reliable hardware control while the workstation handles inference.
Trustworthy Feedback
Human corrections must not be used as training data without review. Tempus separates technician feedback, dataset curation, model training, validation, and release.
Accomplishments
- Booted QNX 8.0 on Raspberry Pi 5
- Enabled live IMX708 camera capture
- Built a native C frame bridge
- Reconstructed full-color NV12 frames
- Connected the feed to a blood-cell YOLO model
- Created real-time detection analytics
- Built an interactive veterinary dashboard
- Designed a technician feedback loop
- Implemented an experimental ST7789 driver
- Reverse-engineered the initial ToupTek protocol
What’s Next
- Finish the technician annotation interface
- Add authenticated reviewer accounts
- Store consented and anonymized specimens
- Add annotation quality checks
- Track dataset and model versions
- Measure model improvement across feedback cycles
- Run ONNX inference directly on QNX
- Complete the AmScope MU1000 driver
- Train on veterinary-specific species and specimens
- Add calibrated micrometer measurements
- Validate results with veterinary professionals
- Export findings into patient records
- Generate PDF reports
- Compare historical specimens
Potential Impact
Tempus points toward an affordable microscopy platform for:
- Faster laboratory documentation
- More consistent cell counting
- Remote collaboration
- Structured veterinary datasets
- Continuous expert-guided improvement
- Better access for smaller veterinary practices
The most important part of Tempus is not that an AI can make predictions.
It is that every prediction can be reviewed, corrected, and transformed into better knowledge.
Tempus is a microscope, an AI assistant, and a learning system built around human expertise.
Responsible Use
Tempus is an experimental decision-support platform.
It does not provide medical diagnoses and must not replace professional veterinary judgment, validated laboratory equipment, or appropriate clinical testing.
Technician feedback should be curated and validated before being used for model training. Patient and clinic information must be protected through appropriate consent, access controls, and anonymization.
Log in or sign up for Devpost to join the conversation.