Inspiration
Visible Light Communication (VLC) sends data using ordinary LED lighting. It is fast, secure, and needs no radio spectrum, but it has a serious weakness: it depends on line of sight. When a person walks in front of the light, or the receiver moves a little, the signal collapses and the bit error rate (BER) shoots up.
An Intelligent Reflecting Surface (IRS) can bounce light toward the receiver and rescue the link, but the surface must be re-configured every time conditions change. I had already worked on a VLC project involving IRS, and I kept running into the same question: who decides how to reconfigure the surface, and when? Doing it by hand doesn't scale, and a fixed script can't reason about messy combinations of blockage, noise, and brightness limits.
What it does
LumiSteer is a web app that simulates an indoor IRS-assisted VLC link and uses an AI agent to keep it stable.
- A physics engine computes received power, irradiance, SNR, and BER.
- An optimizer calculates the best IRS element angles.
- An LLM agent (NVIDIA Nemotron on Nebius Token Factory) acts as the supervising engineer. It reads link telemetry, diagnoses what went wrong (direct path blocked, receiver moved, high ambient light, illumination limit), chooses which tool and settings to use, checks the result against targets, retries with a different strategy if needed, and explains its decision in plain language.
An important design choice: the LLM never computes BER, power, or intensity, and never calculates mirror angles. LLMs are unreliable at numerical work, so every number shown comes from the physics engine and optimizer. The LLM handles diagnosis, strategy, verification, and explanation.
How we built it
Physics model. The line-of-sight channel gain uses the Lambertian model:
$$H_{LOS} = \frac{(m+1)A}{2\pi d^2}\cos^m(\phi)\,T_s\,g(\psi)\cos(\psi), \qquad m = \frac{-\ln 2}{\ln\cos\Phi_{1/2}}$$
Each IRS element reflects light according to the specular law, weighted by how well the reflected beam points at the receiver:
$$H_{IRS} = \sum_{k} H_k \, \rho \, \exp!\left(-\frac{\delta_k^2}{2\sigma^2}\right)$$
where $\delta_k$ is the angular error between element $k$'s reflected beam and the receiver. Received power is $P_r = P_t (H_{LOS} + H_{IRS})$. Noise combines shot and thermal terms:
$$\sigma_n^2 = 2qR(P_r + P_{amb})B + \frac{4kTB}{R_L}, \qquad SNR = \frac{(R P_r)^2}{\sigma_n^2}$$
and for OOK modulation, $BER = Q(\sqrt{SNR})$.
Optimization. We optimize all $2 \times N \times M$ element tilt angles with L-BFGS-B, under constraints (LED power limit, minimum illuminance at the work plane, tilt range). For stability we also run a robust optimization that minimizes the average and worst-case BER over receiver positions sampled within a small radius, instead of tuning for one perfect point.
Agent. Nemotron runs on Nebius Token Factory through an OpenAI-compatible API with function calling. Its tools are simulate, optimize_irs, robust_optimize, sweep, and predict_blockage. The loop is: observe, diagnose, call a tool, read the result, compare to the targets, retry if needed, explain. It is capped at 4 iterations, validates every tool argument, and falls back to strict JSON output if native tool calling fails.
Stack. Python, FastAPI, NumPy, SciPy, and Pydantic on the backend; React, Vite, TypeScript, Tailwind, and Recharts on the frontend. The agent streams its steps to an on-screen console over Server-Sent Events.
Comparison. We compare four strategies on identical scenarios: (A) no IRS, (B) random IRS, (C) IRS optimized once at a fixed point, and (D) agent-driven robust IRS. We report BER heatmaps, outage probability (fraction of positions with BER above $10^{-3}$), mean and worst-case power, and irradiance.
Challenges we ran into
- Deciding what the LLM should and shouldn't do. The tempting design was to let the model "steer" directly. We kept all numerical work in deterministic code and gave the LLM the role it is actually good at: reasoning across messy situations and explaining outcomes.
- Stability, not just peak performance. An IRS tuned for one exact receiver position can fail after a small movement. Robust optimization over a neighborhood of positions fixed this, at the cost of a slightly lower peak.
- Competing objectives. Raising intensity improves the link, but the LED has a power limit and the room must stay at a comfortable illumination level, so these are enforced as constraints.
- Keeping the demo reliable. The physics demo works without the LLM, and a clearly labeled cached agent run covers cases where the API is unavailable.
- Performance. Vectorizing the physics in NumPy made a 40 x 40 position sweep run in well under a second. The slow first request after startup turned out to be import overhead, not a real bottleneck.
What we learned
- LLM agents are strongest as supervisors of reliable tools, not as replacements for them.
- Designing for the worst case across small perturbations matters more for link stability than maximizing a single peak value.
- A verification step (check the tool's numbers against targets, then retry) is what turns a chatbot into an agent.
Results
[Fill in after your live runs. Replace the placeholders with real numbers from your Comparison page.]
- Outage probability: A = []%, B = []%, C = []%, D = []%
- Worst-case BER: A = [], B = [], C = [], D = []
- Mean received power and irradiance improvement of D over A: [__]
All results are simulated. The channel model uses [state your parameter source, e.g. values from our earlier VLC-IRS work and published VLC literature].
What's next
- Calibrate the model with real photodiode measurements from a physical VLC-IRS testbed.
- Add adaptive modulation (PAM, OFDM) and multi-user handover.
- Run a lightweight learned controller for millisecond-scale steering, with the LLM supervising at a higher level.
- Deploy on NVIDIA Jetson for edge operation.
Built with
Nebius Token Factory, NVIDIA Nemotron, Python, FastAPI, NumPy, SciPy, React, TypeScript, Vite, Tailwind CSS, Recharts
Log in or sign up for Devpost to join the conversation.