GistLabs Story Quality

Quality Matters. Poor quality stories disrupt productivity and frustrate teams. GistLabs uses AI to objectively measure story quality, recommend improvements and boost your team’s productivity.

Comment

Inspiration

Initially I was interested in code generation from natural language. However, the code being generated was typically poor. On reflection it was apparent that many of my sample stories were missing fundamental information, that prevented them from being turned into code. During a discussion with a good friend, we rethought this approach. We realised that if the code is of low quality, it likely reflects the quality of the story used to generate it. This was a breakthrough moment. We could actually measure story quality!

I have worked as a Product Owner for years and I have trained numerous new business analysts and product owners. The idea that I could build a system, which would help them generate better stories, by measuring story quality and providing feedback, was very appealing to me.

What it does

As someone who has spent years grappling with the challenges of poor quality requirements, I liked the idea of creating a tool that addresses this head-on. “Story Quality” objectively measures story quality, before coding begins. This innovative tool helps product owners to improve their communication and collaboration, getting stories ready for development. It is a Jira add-in, built with Forge, using AI and employing proprietary rules to read and score stories.

Every issue tagged as a story is individually assessed on its ‘codability’, ‘usability’, and ‘simplicity’. Quality improvement guidance is provided for all words, which are identified with alerts, warnings or recommendations, depending on the severity of the quality concern.

There is also a ChatGPT add in that prompts for guidance. The application works without ChatGPT but this additional information has proven very interesting in the readiness assessment and sparking creative thought.

There are dashboards and project pages that provide useful quality summaries and overviews.

How we built it

A typical sentence contains subject, verb and object that should be kept close together. This is how we learned to write good sentences at school, and it is the best way to write stories too. However, this is not just a language consideration, it is a coding consideration. We write stories in natural language to describe behaviour for an environment or domain. That domain is typically converted by programmers to object oriented code, which describes those objects, their attributes, and how they behave. In other words the Subject, Object and Action (SOA) can be turned into coded classes and methods, which sets out the data attributes and functional relationships (methods) of a system.

Pre-trained NLP models are used to create the sentence hierarchy and assign part of speech tags. GistLabs have trained their own proprietary model to perform object oriented code classifications (Class or Method), where applicable. Proprietary rules are then used to evaluate these machine learned structures and their relationships for quality.

Challenges we ran into

This has been a long running pet project with many lessons along the way. I have iterated through numerous technology designs, different programming languages, and cloud designs. The original version was a web application built with a java back end. It had a very complex set of proprietary rules. This proved extremely difficult to maintain and adapt when rule changes where needed. It needed a deeper understanding of the NLP engines and how they tagged words for Parts Of Speech and Dependency parsing. The java frameworks I was using were heavyweight and the cloud lamdas were slow to launch, causing time outs. Over time I simplified the architecture, and moved to a python back end using simple cloud lambdas that load and run quickly.

It became apparent that nobody wanted another application on their desktop. Especially one that didn't integrate with existing, high quality and pervasive technology, like Jira. So I embarked on building the UI into Jira and adapting it for the Jira components. I use an issue panel with 4 tabs, a project page and some dashboard components. This allows product owners to quickly quality assess every story and also to review quality holistically across all stories.

Accomplishments that we're proud of

The delivery of a functional MVP that I believe can help improve team productivity and improve code quality. I really enjoyed how much simplification I achieved by reducing some complex rules into a nice clean and maintainable design, using machine learning.

What we learned

  • The complexities of Natural Language Processing and all of the tagging available in models like stanza and spacy.
  • The power of Jira, and that I really do like Forge.
  • How to simplify complex architectures into small maintainable functions that have virtually no deployment dependencies, using the power of cloud.
  • Coding is amazing, intriguing and powerful. It can also be mind poppingly frustrating and time consuming.
  • When your back is against a wall, you need great friends!

What's next for GistLabs Story Quality

There are numerous add-ons and functional enhancements that I'd like to include. Some of these are for particular industries such as audit, business consultancy and technology consultancy. There are also opportunities outside technology. In particular risk management and process engineering. These might enable us to embark on new business directions with Atlassian.

Share this project:

Updates