AI SideKick - An Alexa Self Programmable AI Sidekick

AI Sidekick is an Alexa voice assistant that uses a website as it's display and also executor of generated code. It is inspired by AutoGPT. Technically if GPT can generate the code, it can execute it.

Comment

Inspiration

AutoGPT, Alexa Web API

What it does

It takes voice and natural language input and produces code to satisfy tasks or to derive an answer to a problem given by the user. Input is via alexa and potentially can also be given back via voice if there is quick enough turnaround. Many times Alexa output such as APL and and WebAPI (restricted) are very limited in nature. By using a way to "link" a webpage to Alexa, you are now free to display or do anything. By also allowing code to execute in a sandbox, you now empower chat gpt to be able to come out with it's own coded solution. At the very least, AI Sidekick https://bit.ly/aisidekick website can act as a quick and dirty on demand code REPL powered by GPT code generation and convenience of voice.

How we built it

  • VoiceFlow with GPT 3.5 and GPT 4
  • Repl.it express server for socket.io and rapid poc
  • Alexa Voice Assistant

Challenges we ran into

  • Prompt engineering to extract code and to generate only code without the flowery language around
  • Voiceflow / alexa web socket output limitation
  • Time, so much potential so little time

Accomplishments that we're proud of

  • As clean code generation as possible
  • Demo of "preset commands" that can be built in as helpers or standalone execution that can be called by chatting it e.g. "Execute sample command" hence a very crude plugin architecture is born!

What we learned

Sockets, VoiceFlow, Prompt Engineering

What's next for AI SideKick - An Alexa Self Programmable AI Sidekick

If demand is good, i will start a service for:

  • Alexa or any voice dev to have an API to use a website to display real time visuals
  • Voice Driven AutoGPT with simple commands such as generate screenscraping code and pipe it to a email function. Basically a pipeline to execute tasks that are programmable.

Built With

Share this project:

Updates