RefineMeFaster - Custom fields meet AI

Get suggested improvements for your issue description.
Full control over the data send to the AI (e.g. remove sensitive data).
See the number of improvements at a glance.
Full transparency about the AI service, prompts and data.
The data flow of RefineMeFaster

Inspiration

Our journey is deeply rooted in the firsthand challenges we've faced with extensive backlog refinements. Determined to find a solution, we turned to the potential of AI to help teams tackle this. Beginning with Jira as our foundation, we integrated AI to guide Product Owners in refining user stories incrementally, breaking down obstacles, and nurturing an environment for seamless team collaboration. Our ultimate goal is to empower you, drawing from our own struggles, to reduce the time spent on backlog issues and channel your efforts into delivering substantial value to your customers.

What it does

Our app streamlines the process of improving user stories in Jira. Open the custom field dialog to initiate the improvement process. Review and prepare the data for analysis, ensuring sensitive information is removed. AI-powered analysis suggests improvements by examining the issue's summary and description. You decide whether to keep suggestions open (unresolved but valid) or drop them. The number of open suggestions appears as a numeric value in the custom field after saving, helping you track ongoing improvements. You can mark suggested improvements as resolved if you modify the sentences. The open suggestion count supports Jira Query Language tasks like sorting issues by the highest number. It's also usable in automation, triggering actions or workflows if the open suggestions exceed a threshold.

How we built it

Drawing upon our prior familiarity with Forge, we opted for a fresh approach this time around, setting out to explore the potential of the forge-sandbox for launching our project. The experience was both enjoyable and highly effective from the outset.

Our journey unfolded through a series of significant stages:

Exploring New Horizons: We initiated our journey by delving into uncharted territory, integrating Chat GPT into the Forge backend. This crucial step served as the bedrock upon which our AI-powered developments would be constructed.
Creating 'Release Notes' with AI: Our initial undertaking involved a seemingly straightforward task: utilizing AI to generate 'release notes.' This initial endeavor functioned as our proof of concept, laying the groundwork for more ambitious AI integrations.
Venturing into 'Improvement Fields': Having tasted early success, we ventured further into more intricate projects, christened as 'Improvement Fields.' These challenges pushed the boundaries of what AI could achieve within our Forge environment.
Iterative Prompt Enhancement: To achieve precision in our AI-driven solutions, we embarked on multiple rounds of prompt refinement. This iterative process entailed crafting progressively complex prompts to fine-tune the AI's responses.
Elevating User Experience: While backend work remained crucial, we equally focused on enhancing the user interface. Ensuring an aesthetically pleasing and user-friendly design was central to our efforts.
Real-world Testing and Collaborative Insights: Our journey would be incomplete without testing the real-world applicability of our solutions. We conducted extensive testing with actual user scenarios and collaborated with product owners to collect valuable feedback for refinement.

Our project represented a thrilling fusion of Forge's capabilities and the potential of AI. Each step contributed to our overarching goal of delivering a cutting-edge solution. Our story encapsulates the spirit of innovation, cooperation, and iterative development that fueled our success in this extraordinary endeavor.

See the data flow in the image slider of the page

Responsible AI: Data privacy and transparency

Terms of Use (Admin Consent): Admins can see the terms of use of the in use AI framework. Idea: Future development includes detailed terms of use with AI usage, data handling, and user data protection. Admins can review and agree during onboarding.
Prompt display: Admins can see the AI prompts in the admin section. Idea: We're planning an extensive prompt library with guidance for customizing prompts to meet specific needs.
User Data Visibility and Adjustment: Users view and adjust data send to the AI. Idea: Future developments introduce a user-friendly data dashboard, allowing easy adjustments of data-sharing preferences.
Curated Result Selection: Users review AI results and may change the outcome. Idea: Our vision includes allowing users to actively select preferred AI suggestions, streamlining the story refinement process.

These privacy ideas reflect our commitment to a more transparent, user-centric, and privacy-conscious development.

Challenges we ran into

During the development of our Forge app for Jira, we encountered various obstacles. To begin with, we had to contend with the stringent time constraints imposed by Forge, which at times conflicted with the response times of the ChatGPT API. Moreover, integrating AI into Jira while ensuring data privacy and security posed an additional challenge. Another stumbling block revolved around the limitations of the Forge app sandbox and its restricted Node runtime environment. For instance, we were unable to incorporate the 'google-auth-library' to leverage Google Vertex as an alternative to ChatGPT. However, it's worth noting that we've been informed that this issue will be addressed in the near future, which is encouraging for the development of our Forge app.

Working with AI presents several challenges that can significantly impact the outcomes of a project. The initial hurdle we encountered was the difficulty in obtaining the desired results from the AI model. We had to go through numerous iterations, even resorting to refining the prompts using the AI itself to improve the output. This struggle highlights the need for a deep understanding of how to effectively communicate with the AI and the importance of fine-tuning. Another obstacle we faced was related to the format of the input data. Ensuring that the data is correctly preprocessed and structured for the AI model can be a complex and time-consuming task. Incorrect data formatting can lead to misleading or unusable results, underscoring the need for rigorous data preparation.

Accomplishments that we're proud of

We successfully created custom fields within Jira, namely 'Improvements' and 'Release Notes,' which promise to enhance our daily work significantly. These fields streamline the process of summarizing issues and generating release notes, making our workflows more efficient and productive.

Personally, we find immense satisfaction in using these fields and witnessing the AI-generated outputs.

Our project's commitment to transparency is another achievement that we hold in high regard. We've designed our app to empower users to have control over their data. Before any information is evaluated by the AI, customers can fine-tune the input themselves, allowing them to withhold sensitive information, if necessary.

What we learned

The experience of working on this project was a valuable learning journey for our team. None of us had previously connected with a large language model (LLM), but we quickly grasped the potential and simplicity of the OpenAI API. We also gained insights into the importance of data security and privacy regarding Ai, making sure that sensitive information is not inadvertently shared.

Forge

Using a template: the forge-sandbox provides many useful components that make development much easier
New components: we tried some (for us) new Atlassian components. For example the Inline edit, which was way faster than developing something like this ourselves

OpenAI

Prompt Refinement: Emphasize the importance of refining prompts with AI for generating long dialogues.
Example Utilization: Encourage using concrete examples to illustrate both input and output for better comprehension.
Preprocessing:
1. Issue Editor Format: Highlight the challenges associated with issue editor format, which can be hard to parse for AI.
2. HTML: Address the performance issues AI encounters when processing HTML content.
3. Sentence Format: Promote the use of sentences, which are easier for AI to understand and process quickly.
Explicit Details: Stress the significance of providing explicit instructions, including specific grammar preferences, the expected length, and the desired number of results.
Parameter Exploration: Encourage the exploration of various ChatGPT parameters such as temperature, etc.

Sidequest: CSS and SVG

Creative Effects: Mention the opportunity to explore creative CSS and SVG effects, including cool rainbow effects and animations.

What's next for RefineMeFaster

Enhancing Context Understanding: Our primary focus in the next phase is to significantly enhance the AI's capacity to understand the broader context within user stories, moving beyond individual tokens. This critical improvement will result in more comprehensive and contextually relevant suggestions for your user stories, making the entire refinement process more precise and valuable.
Introducing Google Vertex: Google Vertex joins ChatGPT, offering alternative AI capabilities for user story refinement.
Data Structuring Assistance: Users will soon provide templates to help the AI comprehend data better in user stories.
Interactive Description Enhancement: An inline feature enables real-time improvements and suggestions as you work on user stories.
Automated Analysis: Post-creation, automated analysis ensures story quality and consistency.
Performance Enhancement: We're committed to optimizing AI performance for faster, more efficient results in user story refinement. A quicker, more responsive tool is on the horizon to streamline your workflow, and your feedback remains instrumental in this journey.

Custom fields meet AI in general

In the near future, we have exciting plans to expand our project with advanced AI fields. These include estimating effort, assessing technical complexity, and providing detailed task breakdowns for Jira issues. Additionally, we're working on an "AI Prioritization Field" to help teams determine the business value of their tasks. These enhancements will make Jira even more efficient and intelligent for our users.