Typical Topical

A skill for Amazon Alexa using Expert.ai textual analysis that challenges users interactively to meet targets for topic, behavior and emotions in their conversation with Alexa.

Comment

Inspiration

I have been exploring Alexa skills (apps) that are interactive based on sentiment and emotion analysis, not just analyzing utterances after the fact, but involving the user in the process of generating conversation around target sentiments and emotions. With Expert.ai NL API, I discovered I could incorporate topics and behaviors types, along with emotion types, in a more interactive user engagement.

What it does

Typical Topical is a demonstration game that challenges you to say things that meet a target you are given for topic, behavior type and emotion type. You score based on how closely you meet those targets. The skill uses textual analysis from Expert.ai to detect and determine the topic, the behavior described and the emotion expressed in the statements you make.

LEVELS: You can choose Easy, Medium, or Hard, to be given one, two, or three targets respectively. Say 'Easy', 'Medium' or Hard' to select a level.

SCORING In each round, you will be dealt one or more of a topic, a behavior to describe, and an emotion to express. • Topic If Expert.ai detects the exact target topic in your statement you get 10 points. • Behavior If it detects that you incorporated the exact target behavior, you get 10 points. If you incorporate a behavior from the same general group and intensity of the target behavior, you get 4 points. If you incorporate a behavior from the same general group as the target behavior, you get 2 points. • Emotion If it detects that you expressed exact target emotion, you get 10 points. If it detects an emotion from the same emotion group, you get 4 points. Scoring for any category is halved if you use the exact target word (topic, behavior or emotion). Please do not be frustrated if the skill does not detect something that you think should be obvious. It happens. Just as in 'Whose Line is it Anyway'... the points don't matter.

NOTES:

• If you say 'Repeat' after being dealt your targets, you can hear them again. Use this in case you are tongue-tied and need a bit more time to make up your statement. • Sometimes if your statement is too long, Alexa will treat it as though you said nothing at all, and ask you to say something again. Either say something a little shorter, or say 'Repeat' to repeat the targets. • If you say 'Repeat' after you are shown the results, you will get the same targets to try again, but you won't score. Use this to practice. • If you say 'Help', you will hear more details, and receive a card in your Alexa app that has all this information. • If you say 'Goodbye', you'll hear your final score before you leave.

The list of behavioral and emotion traits can be found at https://docs.expert.ai/nlapi/v2/reference/categories/ The list of detectable topics can be found at https://docs.expert.ai/nlapi/v2/reference/topics/

How we built it

I used the Alexa Skills Kit SDK, creating interaction models in JSON and back-end processing as an AWS Lambda function using Node.js. I use the list of Expert.ai-detectable topics, and the hierarchies of behavioral traits and emotional traits to generate targets for the user to meet. After receiving a user utterance from Alexa, I send the utterance in parallel to three Expert.ai NL APIs: analyze to detect a topic, categorize/behavioral-traits to detect behavior, and categorize/emotional-traits to detect emotions. Scoring is based on the ability of the user to utter a statement that meets the targets.

Challenges we ran into

A primary challenge in marrying the Alexa platform to Expert.ai analysis is the limitations of interactions with Alexa. Though the Alexa API does not explicitly enable open-ended utterances (vs those with more constrained forms and identified slots), there are workarounds that get the complete utterance. However, there is a real but indeterminate time limit for utterances, where speaking too long to Alexa is treated by the service as not speaking at all. I believe that textual analysis would be more fruitful if I could process longer utterances than allowed for by Alexa. Alexa also is quick to begin processing on even the slightest of pauses, and in my skill a user is more likely than not to have to pause to think of a suitable utterance to meet the given targets.

There were several challenges to using Expert.ai to not only analyze text but to generate target topics, behaviors and emotions. Some of the Expert.ai-detectable topics are really not conducive to spontaneous utterances (What's there to say about five-a-side football? What is five-a-side football?), so I removed them from the target topic set. On the textual analysis side, there were many occasions where an emotion or behavior that was seemingly expressed went undetected, or where a target topic that was part of a hierarchy returned a parent topic but not the target itself. I may attribute this to the limited size of the utterance allowed by Alexa, but it was a challenge.

Accomplishments that we're proud of

I continue to make progress on devising interactive experiences that incorporate textual analysis into realtime conversations. I also furthered my expertise in the graphical interface that works together with the voice interaction to make an overall more engaging experiences that have devices that support both voice and animation.

What we learned

I learned about the limitations of incorporated external analysis within the constraints of an independent voice interface system, and how to design workarounds that would still lead to a meaningful and engaging experience. I expanded my repetoire of graphics-voice integration, and in programming for parallel requests to an API in order to meet the timing constraints of Alexa's conversational platform.

What's next for Typical Topical

A goal for real-time conversational AI is to generate responses and content to the user that are based on the analysis of categories such as topics, behaviors and emotions. In Typical Topical, interactivity was based on prompting the user to meet targets. In future integration of Expert.ai with Alexa-mediated conversations, I would look to have Alexa's responses, from content to phrasing to vocal expression, informed by the results of the textual analysis.

Typical Topical

Alexa skill to use Expert.ai textual analysis and categorization in an interactive challenge

This repository contains the node.js code for a Lambda backend and a JSON interaction model that can be used to create Typical Topical, an interactive excercise challenging the user to match targets for topic, behavior type, and emotion type by speaking sentences which Expert.ai will analyze and score.

The skill may be implemented by using the ask-sdk cli, or with the Lambda and Alexa consoles. See: https://developer.amazon.com/en-US/docs/alexa/alexa-skills-kit-sdk-for-nodejs/overview.html https://developer.amazon.com/en-US/docs/alexa/smapi/ask-cli-intro.html

Note that the backend code makes use of two environment variables which will need to be provided to the Lambda function: Expert.ai username and password. The skill.json file will need to include an endpoint URI for the Lambda function.

What it does

Typical Topical is a demonstration game that challenges you to say things that meet a target you are given for topic, behavior type and emotion type. You score based on how closely you meet those targets. The skill uses textual analysis from Expert.ai to detect and determine the topic, the behavior described and the emotion expressed in the statements you make.

LEVELS: You can choose Easy, Medium, or Hard, to be given one, two, or three targets respectively. Say 'Easy', 'Medium' or Hard' to select a level.

SCORING In each round, you will be dealt one or more of a topic, a behavior to describe, and an emotion to express. • Topic If Expert.ai detects the exact target topic in your statement you get 10 points. • Behavior If it detects that you incorporated the exact target behavior, you get 10 points. If you incorporate a behavior from the same general group and intensity of the target behavior, you get 4 points. If you incorporate a behavior from the same general group as the target behavior, you get 2 points. • Emotion If it detects that you expressed exact target emotion, you get 10 points. If it detects an emotion from the same emotion group, you get 4 points. Scoring for any category is halved if you use the exact target word (topic, behavior or emotion). Please do not be frustrated if the skill does not detect something that you think should be obvious. It happens. Just as in 'Whose Line is it Anyway'... the points don't matter.

NOTES:
• If you say 'Repeat' after being dealt your targets, you can hear them again. Use this in case you are tongue-tied and need a bit more time to make up your statement. • Sometimes if your statement is too long, Alexa will treat it as though you said nothing at all, and ask you to say something again. Either say something a little shorter, or say 'Repeat' to repeat the targets. • If you say 'Repeat' after you are shown the results, you will get the same targets to try again, but you won't score. Use this to practice. • If you say 'Help', you will hear more details, and receive a card in your Alexa app that has all this information. • If you say 'Goodbye', you'll hear your final score before you leave.

The list of behavioral and emotion traits can be found at https://docs.expert.ai/nlapi/v2/reference/categories/ The list of detectable topics can be found at https://docs.expert.ai/nlapi/v2/reference/topics/

How we built it

I used the Alexa Skills Kit SDK, creating interaction models in JSON and back-end processing as an AWS Lambda function using Node.js. I use the list of Expert.ai-detectable topics, and the hierarchies of behavioral traits and emotional traits to generate targets for the user to meet. After receiving a user utterance from Alexa, I send the utterance in parallel to three Expert.ai NL APIs: analyze to detect a topic, categorize/behavioral-traits to detect behavior, and categorize/emotional-traits to detect emotions. Scoring is based on the ability of the user to utter a statement that meets the targets.

Share this project:

Updates

posted an update —

NOTE: I inadvertently posted my GitHub README.md in my submission rather than the correct copy. Please note the correct copy as follows:

Inspiration

I have been exploring Alexa skills (apps) that are interactive based on sentiment and emotion analysis, not just analyzing utterances after the fact, but involving the user in the process of generating conversation around target sentiments and emotions. With Expert.ai NL API, I discovered I could incorporate topics and behaviors types, along with emotion types, in a more interactive user engagement.

What it does

Typical Topical is a demonstration game that challenges you to say things that meet a target you are given for topic, behavior type and emotion type. You score based on how closely you meet those targets. The skill uses textual analysis from Expert.ai to detect and determine the topic, the behavior described and the emotion expressed in the statements you make.

LEVELS: You can choose Easy, Medium, or Hard, to be given one, two, or three targets respectively. Say 'Easy', 'Medium' or Hard' to select a level.

SCORING In each round, you will be dealt one or more of a topic, a behavior to describe, and an emotion to express. • Topic If Expert.ai detects the exact target topic in your statement you get 10 points. • Behavior If it detects that you incorporated the exact target behavior, you get 10 points. If you incorporate a behavior from the same general group and intensity of the target behavior, you get 4 points. If you incorporate a behavior from the same general group as the target behavior, you get 2 points. • Emotion If it detects that you expressed exact target emotion, you get 10 points. If it detects an emotion from the same emotion group, you get 4 points. Scoring for any category is halved if you use the exact target word (topic, behavior or emotion). Please do not be frustrated if the skill does not detect something that you think should be obvious. It happens. Just as in 'Whose Line is it Anyway'... the points don't matter.

NOTES:
• If you say 'Repeat' after being dealt your targets, you can hear them again. Use this in case you are tongue-tied and need a bit more time to make up your statement. • Sometimes if your statement is too long, Alexa will treat it as though you said nothing at all, and ask you to say something again. Either say something a little shorter, or say 'Repeat' to repeat the targets. • If you say 'Repeat' after you are shown the results, you will get the same targets to try again, but you won't score. Use this to practice. • If you say 'Help', you will hear more details, and receive a card in your Alexa app that has all this information. • If you say 'Goodbye', you'll hear your final score before you leave.

The list of behavioral and emotion traits can be found at https://docs.expert.ai/nlapi/v2/reference/categories/ The list of detectable topics can be found at https://docs.expert.ai/nlapi/v2/reference/topics/

How we built it

I used the Alexa Skills Kit SDK, creating interaction models in JSON and back-end processing as an AWS Lambda function using Node.js. I use the list of Expert.ai-detectable topics, and the hierarchies of behavioral traits and emotional traits to generate targets for the user to meet. After receiving a user utterance from Alexa, I send the utterance in parallel to three Expert.ai NL APIs: analyze to detect a topic, categorize/behavioral-traits to detect behavior, and categorize/emotional-traits to detect emotions. Scoring is based on the ability of the user to utter a statement that meets the targets.

Challenges we ran into

A primary challenge in marrying the Alexa platform to Expert.ai analysis is the limitations of interactions with Alexa. Though the Alexa API does not explicitly enable open-ended utterances (vs those with more constrained forms and identified slots), there are workarounds that get the complete utterance. However, there is a real but indeterminate time limit for utterances, where speaking too long to Alexa is treated by the service as not speaking at all. I believe that textual analysis would be more fruitful if I could process longer utterances than allowed for by Alexa. Alexa also is quick to begin processing on even the slightest of pauses, and in my skill a user is more likely than not to have to pause to think of a suitable utterance to meet the given targets.

There were several challenges to using Expert.ai to not only analyze text but to generate target topics, behaviors and emotions. Some of the Expert.ai-detectable topics are really not conducive to spontaneous utterances (What's there to say about five-a-side football? What is five-a-side football?), so I removed them from the target topic set. On the textual analysis side, there were many occasions where an emotion or behavior that was seemingly expressed went undetected, or where a target topic that was part of a hierarchy returned a parent topic but not the target itself. I may attribute this to the limited size of the utterance allowed by Alexa, but it was a challenge.

Accomplishments that we're proud of

I continue to make progress on devising interactive experiences that incorporate textual analysis into realtime conversations. I also furthered my expertise in the graphical interface that works together with the voice interaction to make an overall more engaging experiences that have devices that support both voice and animation.

What we learned

I learned about the limitations of incorporated external analysis within the constraints of an independent voice interface system, and how to design workarounds that would still lead to a meaningful and engaging experience. I expanded my repetoire of graphics-voice integration, and in programming for parallel requests to an API in order to meet the timing constraints of Alexa's conversational platform.

What's next for Typical Topical

A goal for real-time conversational AI is to generate responses and content to the user that are based on the analysis of categories such as topics, behaviors and emotions. In Typical Topical, interactivity was based on prompting the user to meet targets. In future integration of Expert.ai with Alexa-mediated conversations, I would look to have Alexa's responses, from content to phrasing to vocal expression, informed by the results of the textual analysis.

Log in or sign up for Devpost to join the conversation.