PitStop-Comment Manager for Your Team that Runs on Atlassian

Actionable Insights and Tasks from Comments that you can take bulk actions on for entire team. Spam Detection, Intent Actioning, Reply All, Rovo Summarize and Remind Me, running entirely on Atlassian.

Comment

Inspiration and What we Built

KEY INSIGHT: 💬 Comments are the fastest way for any Business Team to Coordinate. Most Action Items are flimsy tracked through Comments Tagging a User.

We wanted to push Forge to its limits (and we think we have!) by building Comment Intelligence and Actionability right into the Forge platform. We also did not want to step on Rovo's toes (i.e., no replication of functionality already present in Atlassian — summarization / rewriting / suggestions, etc.).

Our Motivations:

  • 🔒 Process entirely on Atlassian both for deterministic and contextual outcomes
  • 🤝 Provide context to Rovo but do not replace any existing functionality
  • ✨ Polish the interface to a joyful shine to ensure user engagement

We were excited about the LLM API to achieve processing; however, processing latency (read below to see our optimizations), token cost (we would have to process bulk comment data that might be tens of thousands), and lack of GA made this a no-go for Codegeist 2025.

Our major code contributions as part of this Hackathon

  • ⚙️ Neural Network engine with weights + model in pure JavaScript (TypeScript source) with about 0.5 MB weights, ~0.45 BCE log loss (~70% accuracy) for CPU processing in 0.065s
  • 🎨 SVG animation library to create dynamic animated racetracks in pure JavaScript (we use this to display usernames as a fun race 🏁)
  • 📊 Dashboard with dynamic data and actions built purely in Forge UI Kit
  • 🤖 Rovo invocation with context generated by our engine in a scheduled process
  • ✍️ Online actions with built-in data updates so you can fix comments immediately

Why we built it

This was because we kept putting up comments on each of our team agendas and those comments mostly got lost in the clutter, all the time, every time. While Page and Space intelligence is extremely well covered by Rovo, there is no real comments intelligence available as of 2025.

Here's a list of missing functionalities we faced in Confluence:

  • ❌ There's no inbuilt classifier that tells us if a comment is urgent or has an intent
  • ❌ There's no !remindme or !reminduser bot that schedules a reminder
  • ❌ There's no spam filter or accidental doxxing prevention
  • ❌ There's no flow of comments, only immediate comments or Rovo summaries
  • ❌ There's no historic insights that we can leverage to reply to a chain on Rovo summaries
  • ❌ Rovo summaries only work on a per page context, not across sources

All this surely does not behoove a High-Performing Team™!

So we thought we'd be golden if we could get the following insights into our comments:

  • [x] Urgency and Intent on comments (not just inline but on footer comments too)
  • [x] Remind me / anyone else about an important deadline in a lightweight manner
  • [x] Assign an unreplied comment to someone ITK
  • [x] Nonsense comment detection (especially on our blogs)

Boy! 😄 Did we have a fun time making this work! Here's our dashboard, fully populated. Dashboard

What it does

It computes insights on comments directly in Forge. Yes, that's correct:
🚫 NO long-running LLM calls, 🔐 NO data privacy concerns, and NO cost for inference minimal cost after Jan '26 😉

We built beautiful dashboards (along with a really special landing page, in the spirit of Teamgeist and Codegeist) to show data and insights and allow anyone to take immediate action on comments.

  • [x] Contextual Comments Dashboard for a single space
  • [x] Global Dashboard for comments across spaces
  • [x] Actionable Items split into MECE exploration of comment data
  • [x] Analysis (Charts and Metrics) / Spam (Redact or Delete) / Active Threads (@assigns or reminders on threads) / Attention (None of the above)
  • [x] Context to Rovo Bot with a Summarize using Rovo for multiple threads and chains
  • [x] Bulk Reply, Delete, Redact (spam), Assign (@actions), Remind (date, @user)
  • [x] Comment parser to generate a !remind bot using the excellent chrono library
  • [x] PII and Abuse detection using deterministic regexes
  • [x] Spam and Intent detection using the above-mentioned neural nets
  • [x] Bulk Comment Parsing at scheduled intervals for caching data

How we built it

We basically ended up building two libraries in pure TypeScript that transpile to JavaScript.
Note: These are not yet generally released and were built specifically for the hack.

  • Inkbow → A user-facing library for creating awesome splash pages from names
  • Logi → A Neural Network library written in TypeScript with rollup to JavaScript that builds on Forge

Dynamic Racetrack Generation

  • 🎨 Pure TypeScript SVG manipulation with color and transition parameters for a feature-complete library
  • ✍️ Hand-edited font (originally based on Hershey fonts) to maintain a continuous path
  • 🧩 Forge Frame Component to render with Product Context at runtime

Here's what it looks like (name is publicName from Confluence User Account fetched from the API):

Splash Page in light mode

And here's what it looks like at night 🌛
Splash Page in dark mode

Model Training

  • 🧠 We built custom RNNs as a small standalone alternative to Transformers (~10× smaller for ~0.5× performance)
  • 🔄 The models are trained directly in PyTorch and then weights are loaded in TypeScript (we built the adapter). This is some pretty incredible work, but it gets even better.
  • 📦 We quantized the models to ~0.5 MB denormalized weights and exported pth to JSON
  • 📎 Incorporated the weights directly into the Forge bundle by chunking them into JavaScript

We used the following datasets from 🤗 Hugging Face:

  • Deysi/spam-detection-dataset → For spam detection
  • stefanlarson/outofscope-intent-classification-dataset → For intent classification

Dashboard Creation

  • 🧱 Forge UI Kit all the way (we used almost all available components, but the 🍩 Donut chart is a particular favorite 🤤)
  • 🖼️ The loading splash is a Frame component with our custom racetrack, where we send context using Forge Bridge
  • 📄 Confluence Space Pages (comments within context of a space) and Global Pages (all comments)
  • 🗄️ Key-Value storage for caching comment data and fast retrieval
  • 🏷️ Comment Content Properties for any acted-on comment
  • ⏱️ Async Events (~900s timeouts) for piping bulk actions that trigger API calls, KVS sets, and deletes
  • 🔄 Full refresh on comment data once every hour and incremental refresh on every dashboard action
  • 📬 Every refresh and action goes into a queue to be processed asynchronously, preventing timeouts and allowing dashboard usage while updates occur in the background
  • 🔔 We listen for and *trigger actions on comments in the platform

We also allow context setting for Rovo chat 🤩. Especially useful for related comments across multiple sources. Live Data

Why is this a BIG DEAL for us

We did not believe we'd be even 50% of the way we are right now. This is because of the significant amount of infrastructure required even before tackling the actual dashboard display.

1) No existing continuous-line SVG font library (Apple Hello is the closest, but it is proprietary-licensed and not continuous-line) 2) No existing Recurrent Neural Network library in JavaScript; Andrej Karpathy's library from 10 years ago doesn't match the latest PyTorch SOTA implementation 3) No existing way to port quantized weights to JSON from .pth 4) No existing deterministic MECE split of comment categories on Atlassian

But we managed to get there in the end, hopefully with clear, concise, easy-to-maintain code that is reusable and implementable. That's not to say we didn't face challenges.

What Challenges we faced

Everything was a challenge 😅 Forge made a lot of the hard parts super easy. As someone who has spent days on AWS Lambda + AppSync configuration with IAM policies and other configuration hell, Forge is deeply appreciated for abstracting much of that complexity.

A small list of challenges we faced with Forge while developing:

  • 📘 Frame documentation is unclear for vanilla JS + Forge Bridge (we had to roll our own build)
  • 🔁 Async Events are not callable directly from the frontend and cannot return data
  • 🎨 Generating SVG in JS is a HARD problem
  • 🧠 Replicating a simple PyTorch RNN in JS is an EVEN HARDER problem

What we learned

👍 Anything is doable with a positive attitude and Forge UI Kit. We mean it literally… literally.

What's next for PitStop-Comment Manager for Your Team that Runs on Atlassian

  • [ ] 🚀 Better caching and refreshing (this can be optimized to be much faster and more dynamic)
  • [ ] 🔍 More sorting and filtering options
  • [ ] 🐛 Bug fixes — so many bug fixes
Share this project:

Updates