Inspiration
What it does
How we built it
Challenges we ran into
Accomplishments that we're proud of
What we learned
What's next for Tipentes## Inspiration
Traditional cybersecurity tools (WAFs, scanners) have a massive blind spot: Business Logic Errors. A firewall sees a network request for "$1.00" and approves it because the syntax is correct. A human analyst, however, looks at the screen, sees the item is listed for "$100.00", and immediately knows it's fraud.
Machines historically lacked this "visual context." They couldn't verify if the code matched the user experience. We built TIPENTES to bridge this gap by giving AI the ability to "see" the application just like a human user does.
What it does
TIPENTES is an autonomous threat intelligence analyst that detects invisible logic vulnerabilities. It operates on a simple but powerful premise: Correlation.
- Ingestion: The user uploads a screen recording of a session (MP4) and the corresponding network logs (HAR file).
- Multimodal Analysis: It uses Gemini 1.5 Pro to process both streams simultaneously. It "watches" the video to understand the user's intent (e.g., "User added a Premium Plan to cart") and cross-references it with the network packets (e.g., "Request sent with
price: 0.01"). - Verdict: If the visual intent mismatches the digital reality, TIPENTES flags a "Logic Gap."
- Reporting: It generates a critical alert, a visual attack graph (using Graphviz) showing exactly where the logic broke, and even auto-generates a Python script to reproduce the exploit.
How we built it
The application is built as a tactical command center using Streamlit (Python).
- The Brain: The core logic relies entirely on Gemini 1.5 Pro. We leverage its massive context window to feed it entire session logs and video files at once.
- The Logic Engine: We engineered a "Zero-Inference" prompt architecture that forces the model to cite specific timestamps in the video and specific line numbers in the HAR file as evidence, reducing hallucinations to near zero.
- Visualization: We parse the AI's reasoning into structured data to render dynamic flowcharts via Graphviz, allowing non-technical stakeholders to visualize the attack path.
Challenges we ran into
The biggest challenge was Temporal Grounding. Early versions of the model would understand the video and the logs separately but struggled to link them in time (e.g., matching a click at 00:05 with a request at 00:05.200). We solved this by creating an intermediate parsing layer that indexes the HAR file timestamps and forces the AI to "sync" its visual attention with the network timestamps before making a judgment.
Accomplishments that we're proud of
We successfully created a security tool that finds vulnerabilities that standard static analysis (SAST) tools completely miss. The ability to generate a working exploit.py script based purely on watching a video feels like science fiction, but with Gemini, we made it reality.
What's next for TIPENTES
We plan to move from "Post-Mortem" analysis to Real-Time analysis. By integrating TIPENTES as a browser extension, it could protect users and audit applications live as they are being used, effectively putting a Senior Pen-Tester in every developer's browser.
Log in or sign up for Devpost to join the conversation.