Darwin

Inspiration

The process of developing new features on a large site often resembles the process of evolution in nature. AB Testing allows multiple versions of the same widget to compete for engagement metrics.

Elaboration:

Inspiration for this project comes from the natural process of evolution. Just as species evolve over time through natural selection, web features can evolve through a similar process. AB Testing serves as a method where different versions of a feature (or widget) are tested against each other to determine which one performs best based on user engagement metrics. This concept mirrors how traits in nature are tested for survival and reproduction, with the most successful traits becoming more common in future generations.

What it does

Using a prompt, the AI system can generate multiple versions of code for that prompt, then allow them to compete with each other based on engagement metrics, and eventually come up with a winner.

Usage:

• Define the new code species in the Darwin site with a prompt, along with engagement metrics and allowed actions for that code.
  • Users can input a description or prompt for the feature they want to create. The system uses this prompt to generate different versions (or "species") of the feature. Users also define the metrics that will measure engagement (e.g., clicks, time spent, conversions) and what actions are allowed for each version of the code.
• Add the Darwin widget in your UI. This widget will render different versions of that code to different users.
  • The Darwin widget is integrated into the user interface of the site. It dynamically displays different versions of the feature to different users, collecting data on their performance.
• Based on your "evolution configuration", Darwin will pick a few winners and keep generating new variants until it finds one that has the best performance.
  • The system uses an "evolution configuration" to determine how to select and generate new versions. It continuously evaluates the performance of each version, retains the best-performing ones, and generates new variants to test, refining the feature over time to maximize engagement.

How we built it

• Convex backend
  • The backend of the project is built using Convex, a platform for building scalable, real-time web applications.
• React Frontend
  • The frontend is developed using React, a popular JavaScript library for building user interfaces, especially for single-page applications.
• Amazon Bedrock for Generative AI
  • Amazon Bedrock, a service for building and scaling generative AI applications, is used to generate the different versions of code based on user prompts.
• DarkLaunchy for Experiments and Feature Flags
  • DarkLaunchy, a tool for managing feature flags and running experiments, is used to control the release and testing of different code versions. It allows for toggling features on and off and conducting controlled experiments with ease.

Challenges we ran into

Elaboration:

• Defining the idea took a while.
  • It took considerable time and discussion to settle on the concept of using evolutionary principles for feature development and testing.
• Then it was a bit difficult to set up Amazon Bedrock to run with a simple configuration for the demo.
  • Configuring Amazon Bedrock to work smoothly for the project’s needs and ensuring it could generate the required code variations presented some technical hurdles, especially given the time constraints of a demo.

Accomplishments that we're proud of

Elaboration:

• Still working on it ....
  • As the project is still in progress, specific accomplishments are not listed yet. However, the team can take pride in their progress and the innovative approach they are taking to feature development and testing.

What's next for Darwin

Elaboration:

• Depends on how well it goes on the hackathon ...
  • The future steps for Darwin depend on the outcomes of the hackathon. If the project demonstrates strong potential and receives positive feedback, it could lead to further development, additional features, and potential real-world application. The team is likely to assess the results and decide on the next steps based on their performance and any feedback received during the event.

Built With

• bedrock
• convex
• launchdarkly
• typescript