Vocoder & AI Adventure

• 

  AI Adventure

• 

  Vocoder

Inspiration

Vocoder: Many years ago when I broke my wrist, I was looking into hands-free vocal coding solutions, and they all seemed too wordy and slow. However Copilot & GPT-3's abilities to generate code from descriptions, and do it in a file-aware way makes it so voice input coding can be fast again.

AI Adventure: Adventure games (such as Money Island etc.) always have interactive conversations, but the user is always limited to choosing one out of 4 or 5 options. Using language models, we should be able to make them completely open-ended. It's a messy task for sure, but I think it can be quite fun.

What it does

Vocoder: Press and hold shift to talk into your microphone. When you let go, your Visual Studio Code editor will execute your instructions on your currently open file. You can navigate the file, select lines, find or replace text, or insert code snippets. Describe the task as best as you can and hopefully the right actions will be taken.

AI Adventure: NPCs are now chat bots. Talk to them to try and extract information that is useful in the game. If they resist, you may need to get creative to convince them.

How we built it

Vocoder: Whisper + GPT-3 + VS Code Extension API

AI Adventure: GPT-3

Challenges we ran into

Edge cases are dime a dozen. Extensive testing is required to make these applications polished enough for general use. Language models are hard to tame!

Accomplishments that we're proud of

Quick turn-around. Vocoder was 3 days of work, and AI Adventure was 1 day of work so far.

What we learned

There still are endless fun applications for language models. The key blocker is cost. As cost comes down, it will pay off to have working applications ready for wider release.

What's next for Vocoder & AI Adventure

Excited to get some feedback from other enthusiasts & go from there.

Built With

• flask
• gpt-3
• javascript
• python
• typescript
• vscode
• whisper