Codex with voice

Inspiration

You want Codex to be your buddy, not something on your laptop. You want to call its name and discuss something or assign something with it, without the need to look at the screen.

What it does

Your device would permanently monitor human voice with the wake-up word "Codex" with faster-whisper. It ends automatically when no human voice is detected for some reasonable seconds.

Your voice would be transcribed and submitted to Codex CLI.

How we built it

It's a simple tool so one session in Codex is fine. Need to make sure the project is modularized for sure.

Challenges we ran into

1. Codex is always playing safe, sometimes missing the better solution in detecting whether the voice ends
2. It failed to debug when we ran into an authorization issue
3. It reads out codes and paths, and it sounds so awkward.

Fix:

1. Personalize a skill to have Codex work more aggressively. Basically, encourage it to find a better solution online or within the open-source repos.
2. Make sure the modules are nicely separated and design the approaches for debugging myself.
3. Add a prefix prompt to have Codex only output the result in plain words.

Accomplishments that we're proud of

Haha it's just a simple project. Will let you know once it managed my comic pages with care

What we learned

Always keep and update a universal skill to avoid Codex playing safe when the solution is obviously simple. Sometimes Codex is not "not capable" but always wants to do things meticulously. It's not a bad thing when doing real projects but quite annoying when doing demos.

What's next for Codex with voice

1. It should have the capacity to decide whether it should keep going within the same thread or create a new thread! It's definitely crucial.
2. It should stop reading the code. Not a big deal, as the tool is mostly for non-tech users with daily work. Tech people probably would not rely too much on voice messages.

Built With

• codex