WhyKaigi

Context-aware, near-real-time meeting translation with evidence-linked decisions and action items.

Comment 3

Inspiration

I live in Japan. My Japanese is enough for everyday life, but it is not enough for business meetings.

During a meeting, I could recognize some words and understand pieces of the context. However, I often could not tell why the meeting was happening, how far the discussion had progressed, or what had actually been decided. I usually understood the full picture only after the meeting ended and I used AI to summarize it. That was frustrating because the information was most useful while the meeting was still happening.

I started building WhyKaigi to solve that problem for myself. It helps me follow what people are discussing and see decisions while the meeting is still in progress. Meetings used to feel long and frustrating because I could not fully follow them. Now I actually look forward to meetings, because something I built is helping me understand what is happening in real time.

The name comes from my existing Why series. I first created Why BJT(https://why-bjt-study.vercel.app/), a web application for studying business Japanese for the BJT exam. “WHY” stands for “What Holds You Back?” I chose it after coming to Japan, when I struggled to adapt to the culture and often felt carried along by circumstances. The name was a reminder to overcome what was holding me back.

“Kaigi” means “meeting” in Japanese, so WhyKaigi reflects the question at the heart of the product: what is holding you back from fully understanding the meeting?

WhyKaigi extends that idea from studying Japanese into understanding meetings. My longer-term goal is to make the Why series a place where people from other countries living in Japan can find the tools they need for language learning, meetings, and everyday life.

What it does

WhyKaigi is a Windows-first multilingual meeting companion that combines near-real-time captions, translation, terminology-aware correction, and an evidence-linked Decision Radar.

It can capture system audio from meeting applications or microphone audio and show partial captions immediately. Only finalized speech segments are sent to translation and analysis, so unstable partial text does not create duplicate translations or false meeting records.

The Context Engine lets the user approve important names, company terms, and aliases. Approved terms can improve transcription prompting, deterministic source normalization, translation context, and Radar analysis while the original STT result remains available for audit.

The Decision Radar continuously extracts only explicit:

decisions action items, owners, and deadlines unresolved questions terms or statements that need confirmation

Every Radar item must cite existing finalized source segment IDs. A user can click the evidence, return to the original words, and approve, edit, or delete the suggestion. Changes are preserved in an append-only review history instead of silently replacing earlier results.

WhyKaigi also provides native caption and Radar windows, a one-line media caption mode, Korean/English UI, local session recovery and export, and Google-verified read-only links with room-scoped relay access auditing for meeting participants. Translation or Radar failure never stops the source transcript.

How we built it

The desktop application uses Python 3.11, FastAPI, WebSockets, and an optional Electron host for native transparent, always-on-top windows.

The final Build Week replay pipeline is:

Deepgram Nova-3 transcribes a consented fictional Korean meeting recording. The Context Engine applies only user-approved people and terminology corrections. Gemini 3.1 Flash Lite translates finalized Korean segments into English. GPT-5.6 Luna produces the English Decision Radar. A strict evidence validator rejects items that do not point to real finalized source segments.

The application also supports local faster-whisper transcription, Gemini or OpenAI translation, and an optional local M2M100 CTranslate2 translation worker. Provider calls use separate bounded queues, and results are joined to captions by segment ID so delayed responses cannot attach to the wrong text.

The participant viewer is a React/TypeScript application with a serverless relay and Drizzle-backed room storage. Shared rooms expose only captions, translations, Radar items, and evidence created after sharing starts. Audio, API keys, device information, previous meetings, and host credentials are never shared. Participants sign in with Google through Supabase Auth using PKCE, explicitly continue into each room, and exchange the resulting token with the relay. The relay validates that token server-side before issuing a room-scoped Secure, HttpOnly session cookie. Supabase is used only as the identity provider; WhyKaigi stores no custom attendee-profile or room-audit tables there. The authoritative room audit remains in D1 and is deleted with verified identity, viewer sessions, and relayed meeting text when sharing stops or expires. On an explicit stop, the host receives a sanitized final audit snapshot and may retain that local copy for up to 30 days. Retired email-OTP endpoints return HTTP 410 unconditionally.

How I used Codex and GPT-5.6

I used Codex as an implementation and verification partner throughout Build Week. I brought real product observations—slow subtitle appearance, broken sentence boundaries, duplicate partial/final output, Radar items disappearing, provider errors, and difficult in-meeting navigation—and worked with Codex to turn each observation into a testable contract.

Codex helped trace latency across audio capture, Deepgram boundaries, translation queues, and UI rendering; harden reconnect and failure behavior; build the Context Engine; design and validate evidence-linked Radar output; preserve session integrity; implement participant sharing; and create a sanitized, keyless public replay.

GPT-5.6 is used for the final Decision Radar demo. Its output is never treated as ground truth: every item must cite source evidence and remains reviewable by the user.

The final regression suite reached 369 passing tests with 3 explicitly gated live-provider tests skipped. The public replay uses sanitized events from a real application run synchronized to a separate consented recording, allowing judges to test the core experience without API keys.

Challenges

I did not experience the usual frustration of being blocked by code, because Codex worked through the implementation problems with me. My role was different: I repeatedly used the application, identified where the result did not match what I intended, explained the problem concretely, and defined the level of output I expected. Building with Codex has become part of my everyday life, almost like a hobby.

The one thing that made me hesitate was the cost of using paid APIs. I had never used these services before, so I did not know whether the results would justify the cost. Once I tried them, the range of features I believed I could build expanded significantly, and the quality of the application improved. That was also the point when I first thought, “I can actually use this.”

The most important product challenge was deciding when speech should be finalized and sent for translation. Pauses and sentence lengths differ between Japanese, English, and Korean. Finalizing too early produces fast but disconnected translations. Waiting too long preserves context but makes the captions feel slow. I had to keep testing where that balance should be for each language.

Accomplishments that I am proud of

I am most proud of the translation quality and speed.

From the beginning, I wanted WhyKaigi to feel near-real-time, with the translated caption appearing as close as possible to the conversation currently taking place. I tested different APIs repeatedly and compared not only their translation quality, but also how quickly the result appeared during an actual conversation.

The timing of transcription finalization became just as important as the translation model. Simply cutting speech into short pieces made the translation faster, but it often broke the connection between the previous and following words. WhyKaigi therefore uses language-specific speech boundaries and waits for a more natural point before creating the finalized segment.

It also does not depend entirely on one API result. When a Deepgram segment appears unreliable, WhyKaigi can selectively use the local Whisper model to recheck the risky part while allowing normal segments to continue through the faster path.

The moment WhyKaigi stopped feeling like a prototype came during a meeting test. A finalized English sentence was translated into Korean in under one second, and the result matched the meaning I had heard. A coworker who had previously described the translation as slow and unreliable saw the same result and said that it finally felt ready for a real meeting.

What I learned

I had already built projects with Codex before WhyKaigi, but the change I felt moving from GPT‑5.5 to GPT‑5.6 was substantial, even to someone like me who is not an AI expert.

With GPT‑5.5, there were still many cases where the result did not reach the level I expected, even when I selected a high reasoning level. During this project, GPT‑5.6 followed my intent more consistently and often filled in important gaps that I had not known how to explain. Before this experience, I believed that building a serious product still required a certain level of development and domain expertise. Now I feel that AI can help cover much more of that gap.

My process has also changed. When I have an idea, I first brainstorm with ChatGPT 5.6 and discuss whether it is technically realistic and whether it could become a real commercial product. When the idea still feels convincing after that discussion, I continue the work with Codex and turn it into something I can actually test.

I have always considered myself an action-oriented person, but that used to mean quickly deciding what I could and could not do, abandoning what seemed impossible, and concentrating only on what felt achievable. Since working with Codex, I no longer begin by asking whether an idea is beyond my ability. I focus instead on how it can be made real and how it could reach actual users.

This project taught me that implementation is no longer the first barrier it once was for me. The quality of the idea, product judgment, real-world testing, and the ability to reach users have become more important than ever.

What's next for WhyKaigi

Build Week is still in progress, but I have already started sharing WhyKaigi with people at my company. My first priority is to keep using their feedback to improve the translation quality and near-real-time speed in actual meetings.

After that validation, I want to turn WhyKaigi into a paid product. I previously thought that building a business around one of my own ideas was only a dream. However, seeing even a small number of people purchase and use Why BJT changed that belief. It showed me that this is not an unreachable dream, but a business I may be able to reach if I continue working on it.

Once WhyKaigi becomes a sustainable product, I want to think about which problems people need solved next and continue expanding the Why series. My first goal is modest but meaningful: I want the Why series to become something that Korean residents in Japan naturally mention and recommend to one another. From there, I hope to grow it into a broader platform for people from other countries building their lives in Japan.

I am grateful to OpenAI for giving someone without a traditional development background enough confidence and momentum to participate in a hackathon like this. I am entering Build Week not only to submit an application, but also to prove to myself that the ideas I once left unrealized can now become real products.

Built With

Share this project:

Updates