Project requirements and constraints.

Technical Research

Defining problem space, Finding a market / Market Competitors

One of the lightning demos I presented was a mobile application solution that exists for iOS devices called Endless. This product presents a simplified user interface for creating electronic music and allows others to join a live chat room and collaborate with you in your music-making session. Endless presents a way of collapsing several music track layers into a limited mobile screen size experience, and a real time experience to collaborate.

A blog post made about endless mentions, “thanks to social distancing measures, online collaboration is more prominent than ever. Endless applies to the situation of concern for our project since they have found a medium in which users can collaborate with reduced latency, especially during the coronavirus pandemic of 2020. Two users begin collaborating by one party creating the project and sending an invite to the other to join. On acceptance of the invite, the users can begin cloning versions of the project and requesting updates to the master track that they collaborate on. In scenarios when requests must be approved, users can discuss in the messaging chat room built into the product and discuss what the outcome ought to be. In the end they have a final version of the music they have collaborated on.

Lighting Demos / Sprint Questions & Google Sprint Work

One of our sprint questions was, “How will we know if this is successful?”. A feature that I added into my sketch was to allow the game developer to preview a snippet of their game while experimenting with the music created by the genetic algorithm system. While tweaking the parameters of the system, the game developer will be able to see how the changes in the music will affect the atmosphere that the video game scene creates in union with the soundtrack. This is done in order to provide the user a quick way to determine the success of their 8-bit music soundtrack created for a specific scene in their video game. In order to do so, the game developer can upload recorded video files of the gameplay from their game. This feature was voted for most on the heat map since the ability to preview the gameplay while hearing the music that could potentially be used allows the users to see the combination and determine whether or not the music is the correct fit. This feature was voted for most because the visual queue and the audible queue could be in centralized platforms for both the music creation and the video to be used, allowing for further scaled solutions such as making the sounds generated by the algorithm reflect actions or changes within the actual gameplay/video game clips.

Another feature in the solution that was voted for the most was the ability for users to edit the already genetically generated music for smaller preferences. This feature was voted for because it fulfills the situation of concern, where the user does not have to be an expert to generate music, however has the opportunity to explore ways in which they can tweak the output of the genetically generated music. This gives more end result or output control for their track.

Lastly, another feature that was voted for in my solution was the settings page in which the parameters for the fitness function used to genetically generate the music could be edited. The ability to change how much certain attributes such as distance from each note in the track or how often notes are played loud, allows for more autonomy and control from the user’s perspective. Using standard nomenclature such as minimal and maximum to represent a low or high impact as the potential values for these parameters also simplifies the ability to contribute to the genetic algorithm’s results without a user having to invest in learning much about the scale of the parameters.


Wireframing was completed in Figma. Image is attached.

Usability Testing, Interviews

Interviews agreed to allow their names to be used in documentation.

Interviewed a total of 5 people. We tested our prototype by seeing our interviewees’ screen and them interacting with our wireframe walkthrough created in Figma. My personal notes were created by dividing our low fidelity prototype walkthrough wireframes into sections. I then documented any key phrases or vocal reactions that our user had, as well as the response to the questions asked by the interviewer.

Our test group was divided into 3 categories of expertise: Those who were unfamiliar with video games and music. (Hajung, Pascal) Those who were familiar with games and music, however, were not experts but have experience with creating user interfaces (Kimberly) Those who were experts, either in the mobile game development or in creating music (Nav, Amy)

For the first group of testers, I had noted that both individuals struggle to understand how the metrics were defined in the custom mood selection screen. The design was seen to have some discoverability issues in the suggested instruments and sounds section of the screen. Hajung had a difficult time understanding and deciphering what the end result would actually be and what SoundLab would continue to keep asking of her next. Key phrases like, “what?” or “Ummm I don’t know what is happening” were stated by Hajung several times.

The second category of testers was our classmate, Kim, in the class of Systems Design Engineering 2022. Kim presented a very analytically focused review. She mentioned that our stress-energy map when selecting a custom mood for music was not a real reflection of our nodal map which was in our final overview screen. The nodal map presented a reflection of stress and energy values that varied with time, and we were told that this was confusing since the custom mood map selection had a more static approach of selecting a single (x, y) position which corresponded to a (stress, energy) point.

The last category included individuals with more advanced expertise in the domain of music creation and video game development. Amy has been playing and learning the piano from different certified piano instructors in Kitchener, Ontario. A specific feature that she wanted us to consider as having the ability to choose which notes from instruments count be used. She felt that SoundLab lacked the autonomy or the control that a musically inclined individual would want. She mentioned this by answering our question: What do you expect to happen next after choosing an instrument? Her response included how she expected to choose specific notes and sounds from that instrument. In general, her realization may be alluding to a bigger concept of fearing the unknown. As an individual progresses through our narrative or “wizard” feature flow, they may find themselves getting deeper into the layers of SoundLab without seeing immediate results. According to Neilsen Norman’s design principles, our prototype lacked visibility over the system status.

Improvements (3 Noted - Design Feedback Loops)

1 ) Date: 7/6/2020 Short name: Match Mood or Match Target Insight: In SoundLab Version 3, one of our major changes was to the genetic algorithm that generated music for the user. Our algorithm’s fitness score results varied with a user input of what the priority of the generation ought to be, either matching the target track or matching the mood. This analog control allows users to choose a ratio between how important it is to them for the music generated to match the target track that they create against matching the mood specified earlier in the flow. The insight from the slider that controls this feature is that there is one clear region that if the ratio is set inside, the effects are not distinguishable to the user’s audible senses. Evidence: When our user, Nav set the slider for his output to be focused entirely on mood, he was able to recognize and validate that the music was scary and suspenseful due our algorithm’s mood matching fitness score. Nav did not notice a clear change in the mood when he set the ratio to 0:100, with 0 significance given to matching the target track he had created, and 100 percent focus into matching the mood that he wanted. He mentioned the output midi notes were not reflecting eerie music or rhythmic chaos. In contrast, Nav also could clearly tell that the algorithm had a perfect accuracy in replicating the target track he was creating. Our program was implementing a genetic algorithm that had a 100:0 ratio, where the significance was to the target matching scoring system for the fitness function. Now that only one extreme had shown to be functional, he notified to the interviewers that ratios such as 50:50, 40:60, 30:70, 60:40, 70:30 or 0:100 seemed like the generated music was not presenting an accurate representation of the valence and arousal that was set for our mood by the mood picker in our soundboard. Recognizing mood music requires 57.3% for arousal and valence on a mood map, where in our case no combination or percentage was effective. According to our feedback survey, both users mentioned that they agreed that the music produced could not be interpreted easily in various ratios of target track focus against mood matching focus. The soundboard also caused anxiety since the algorithm never proved an arousal/valence combination percentage in a threshold of 30% above or below 57.3% according to Thayer’s mood model. Recommended Action: The recommendations actions are to explore how to improve the genetic algorithm’s fitness score calculation for a specific mood chosen. Currently the two main criteria that are being taken into account for each randomly generated gene or 16 bit string of music per track is note density, period and a variance between the midi notes used for each track. Another change is to show the output of the genetically generated music since the users did not appreciate hearing the chaotic music generation, further supported by how 47% of shoppers expect a mobile web page to load within two seconds, 40% abandon the site if their wait exceeds three seconds, and each additional second of loading time results in a 7% reduction in conversion.

2 ) Date: 7/6/2020 Short name: Soundboard anxiety Insight: Our user testing for deliverable two did not originally implement a soundboard, which users indicated they felt a lack of control over what could be the output as a musical track for their video game. In our design feedback loop we decided to test a soundboard that allows users to create music according to their judgement while still gaining assistance from a genetic algorithm that will be able to match the music that the user created and converge to the mood they specified according to our preset criteria of each mood. Our soundboard was confusing and difficult to understand for two of the users we tested with in our design feedback loop process. Evidence: One of our users, Nav, did not interact with the soundboard until 80 seconds in upon opening the interface. The boolean toggling process for each note within a bar representing our screen for music was a “mess” according to our user. Additionally, our software was programmed to detect clicks in the generated track. Nav had clicked 6 times, as seen in the video of the interview and as detected by our software’s variable count, emptyClicksForTesting. These clicks further adhere to the amount of self discovery that our software required for a user to understand our soundboard interface. This issue with discoverability also relates to the colours we chose for our soundboard’s track. Our second user interviewer in this design feedback loop, Pascal, mentioned that because the colours of the target track were grey, that the track was locked and was led to spending the next few minutes figuring out how to unlock the track. Our colour choice of grey was chosen to show the user that they have not selected beats in that track currently, and that they may be able to, however as seen in the user interview our intention was not clear for Pascal. Recommended Action: The user suggested that any indication explanation for what is possible and what each musicalical instrument track represents would be helpful. Our action for the next feedback loop is to explore a method of clearly explaining to our user that there is a target track in which they can create music by selecting the individual boxes, representing one beat, and that a generated sequence of beats will be outputted below based upon the mood and target matching algorithm. Team 7 will explore solving this discoverability issue by implementing tooltips in our interfaces that will appear when the user explores features with the application.

3 ) Date: 7/6/2020 Short name: Suggested Instrument Failure Insights: Another major change between our previous design flow and our new one is that we have implemented a different want to reveal suggested instruments and tempo based on mood selection. Originally our instruments were presented in a vertical stack, outlining the suggested one in an accent colour. Another issue we had was users had to switch between pages to add or remove an instrument. Due to our previous deliverable 2 feedback, we decided to tackle both of these concerns in one flow, reducing the steps to add and remove instruments from 3 to 1 and presenting the suggested instruments in a different manner with more control. We used a layout that not only allows the user to confirm or delete the suggested instruments we provide but also change the volume associated with the instrument. When a user proceeds from this page, the instruments confirmed will be visible as a single track in the next interface screen which is the soundboard. From the testing in this feedback loop, both of our users struggled to understand that they couldn’t add any other instruments besides the suggested ones we provided. Evidence: We asked users in our feedback survey and the two users we recorded interviews with, the question: How would you add another instrument?. Both Nav and Pascal, made assumptions that maybe there would be a “+” icon for an instrument to indicate to add another instrument, just as there is a “x” icon to remove an instrument. Recommended action: Though users expect an option to add instruments, our product won’t allow that in order to keep the music generated by the algorithm closely aligned to the mood. In this case, we will explore other indicators on allowing the user to understand they cannot select other instruments besides the suggested ones provided. Our change will exist in the form of showing all instruments that SoundLab can use, however showing the ones not suggested with a grey highlight and a lock symbol to represent how a user cannot use certain instruments due to the mood they selected. This change as well will allow for more discoverability since users will have a visual cue to understand why other instruments are not available, as norman's principles suggest.

Built With

Share this project: